Question

当前的问题是使用python以表格形式解析特定数据。一小部分数据如下所示

Statistics indicator:0x222235

number of records = 3 

records[0]

value one = 2

value two = 5

blocks = 2

block[0] {

some irrelevant data....

value three = 4 bytes

}

block[1]{

some irrelevant data...

value three = 6 bytes

}

records[1]

value one = 3

value two = 5

blocks = 1

block[0] {

some irrelevant data....

value three = 4 bytes

}

records[2]

value one = 7

value two = 6

blocks = 2

block[0] {

some irrelevant data....

value three = 3 bytes

}

block[1]{

some irrelevant data...

value three = 4 bytes

}

Statistics indicator:0x135256

number of records = 2 

records[0]

value one = 4

value two = 8

blocks = 1

block[0] {

some irrelevant data....

value three = 6 bytes

}

records[1]

value one = 3

value two = 5

blocks = 1

block[0] {

some irrelevant data....

value three = 3 bytes

}

如图所示，数据具有特定的模式。它在每个特定数据块的开头都有统计指示器。它具有记录数字段以指示数据块具有的记录数。在每个记录中，``值一''和值二``是不同的。但是，取决于每个记录具有的块数，有几个“值三”，这些值由“块”字段指示。这里的问题是将以下数据以表格形式排列，并添加与特定记录相对应的所有值三。

决赛桌应该像这样：

一值二值三

2          5        10 

3          5         4

7          6         7

4          8         6

3          5         3

我正在考虑的方法是首先找到“统计指标” 如果找到“统计指标”，我将寻找记录的数量和与每个记录对应的块以遍历这些块，然后将值三加起来对应于相似的值1和值2。

这是我尝试提取值一，值二和值三的代码。我还没有研究求和值三。

import re
import pandas as pd
val_dict = { 'value_one':re.compile(r'value one = (?P<value_one>.*)\n'),
           'value_two':re.compile(r'value two = (?P<value_two>.*)\n'),
           'value_three':re.compile(r'value three = (?P<value_three>.*)\n')}

def _parse_line(line):


    for key, val in val_dict.items():
        match = val.search(line)
        if match:
            return key, match
# if there are no matches
    return None, None


def parse_file(filepath):
    data = []  
    with open(filepath, 'r') as file_object:
        row = {}                                # prepare an empty row
        for line in file_object:
            key, match = _parse_line(line)
            # search for keys in the line
            if key == 'value_one':
                value_one = match.group('value_one')
                value_one = int(value_one)
                if 'value one' in row:          # we always have a full row
                    data.append(row)            # append it to the data liest
                    row = {}                    # and reset it
                row['value one'] = value_one    # we have a match: store the value in row

            if key == 'value_two':
                value_two = match.group('value_two')
                value_two = int(value_two)
                if 'value two' in row:
                    data.append(row)
                    row = {}
                row['value two'] = value_two

            if key == 'value_three':
                value_three = match.group('value_three')
                value_three = int(value_three)
                if 'value three' in row:
                    data.append(row)
                    row = {}
                row['value three'] = value_three

        if row != {}:                      # do not forget the last row
            data.append(row)
        data = pd.DataFrame(data)
        return data
if __name__ == '__main__':
    filepath = 'test3.txt'
    data = parse_file(filepath)

Answer 1

在这里，也许我们不想使用正则表达式。但是，如果我们愿意，可以将属性名称设置为左边界，并收集所需的数字，也许使用类似以下表达式：

value\s+(one|two|three)\s+=\s+([0-9]+)

然后，可以将其余问题编写成脚本。如果需要的话，我们还可以在表达式中添加更多边界。

Demo

测试

# coding=utf8
# the above tag defines encoding for this document and is for Python 2.x compatibility

import re

regex = r"value\s+(one|two|three)\s+=\s+([0-9]+)"

test_str = ("Statistics indicator:0x222235\n\n"
    "number of records = 3\n\n"
    "records[0]\n\n"
    "value one = 2\n\n"
    "value two = 5\n\n"
    "blocks = 2\n\n"
    "block[0] {\n\n"
    "some irrelevant data....\n\n"
    "value three = 4 bytes\n\n"
    "}\n\n"
    "block[1]{\n\n"
    "some irrelevant data...\n\n"
    "value three = 6 bytes\n\n"
    "}\n\n"
    "records[1]\n\n"
    "value one = 3\n\n"
    "value two = 5\n\n"
    "blocks = 1\n\n"
    "block[0] {\n\n"
    "some irrelevant data....\n\n"
    "value three = 4 bytes\n\n"
    "}\n\n"
    "records[2]\n\n"
    "value one = 7\n\n"
    "value two = 6\n\n"
    "blocks = 2\n\n"
    "block[0] {\n\n"
    "some irrelevant data....\n\n"
    "value three = 3 bytes\n\n"
    "}\n\n"
    "block[1]{\n\n"
    "some irrelevant data...\n\n"
    "value three = 4 bytes\n\n"
    "}\n\n"
    "Statistics indicator:0x135256\n\n"
    "number of records = 2\n\n"
    "records[0]\n\n"
    "value one = 4\n\n"
    "value two = 8\n\n"
    "blocks = 1\n\n"
    "block[0] {\n\n"
    "some irrelevant data....\n\n"
    "value three = 6 bytes\n\n"
    "}\n\n"
    "records[1]\n\n"
    "value one = 3\n\n"
    "value two = 5\n\n"
    "blocks = 1\n\n"
    "block[0] {\n\n"
    "some irrelevant data....\n\n"
    "value three = 3 bytes\n\n"
    "}")

matches = re.finditer(regex, test_str, re.MULTILINE)

for matchNum, match in enumerate(matches, start=1):

    print ("Match {matchNum} was found at {start}-{end}: {match}".format(matchNum = matchNum, start = match.start(), end = match.end(), match = match.group()))

    for groupNum in range(0, len(match.groups())):
        groupNum = groupNum + 1

        print ("Group {groupNum} found at {start}-{end}: {group}".format(groupNum = groupNum, start = match.start(groupNum), end = match.end(groupNum), group = match.group(groupNum)))

# Note: for Python 2.7 compatibility, use ur"" to prefix the regex and u"" to prefix the test string and substitution.

演示

const regex = /value\s+(one|two|three)\s+=\s+([0-9]+)/gm;
const str = `Statistics indicator:0x222235

number of records = 3

records[0]

value one = 2

value two = 5

blocks = 2

block[0] {

some irrelevant data....

value three = 4 bytes

}

block[1]{

some irrelevant data...

value three = 6 bytes

}

records[1]

value one = 3

value two = 5

blocks = 1

block[0] {

some irrelevant data....

value three = 4 bytes

}

records[2]

value one = 7

value two = 6

blocks = 2

block[0] {

some irrelevant data....

value three = 3 bytes

}

block[1]{

some irrelevant data...

value three = 4 bytes

}

Statistics indicator:0x135256

number of records = 2

records[0]

value one = 4

value two = 8

blocks = 1

block[0] {

some irrelevant data....

value three = 6 bytes

}

records[1]

value one = 3

value two = 5

blocks = 1

block[0] {

some irrelevant data....

value three = 3 bytes

}`;
let m;

while ((m = regex.exec(str)) !== null) {
    // This is necessary to avoid infinite loops with zero-width matches
    if (m.index === regex.lastIndex) {
        regex.lastIndex++;
    }
    
    // The result can be accessed through the `m`-variable.
    m.forEach((match, groupIndex) => {
        console.log(`Found match, group ${groupIndex}: ${match}`);
    });
}

RegEx

如果不需要此表达式，可以在regex101.com中对其进行修改或更改。

RegEx电路

jex.im可视化正则表达式：

将文本文件解析为表格数据以进行处理

1 个答案:

Demo

测试

演示

RegEx

RegEx电路