Question

我正在尝试读取一个大日志文件并进行解析。日志文件包含混合数据类型（示例文件.log.txt）并提取每个类别的最小值和最大值。

log.txt

header: 
seq: 21925
secs: 1603441909
nsecs: 503731023
data_max: 20.0
data_a: [inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, 5.611999988555908, 4.644999980926514, 4.689000129699707, 4.7179999351501465, 4.765999794006348, 4.789999961853027, 0.003000000026077032, 0.001000000026077032, 0.003000000026077032]
data_b: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, inf, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 387.0, 341.0, 0.0, 0.0, 0.0, 0.0, 441.0, 300.0, 302.0, 911.0, 320.0, 334.0, 346.0, 354.0, 359.0, 360.0, 397.0, 418.0, 348.0, 344.0, 342.0, 340.0, 334.0, 333.0, 326.0, 323.0, 322.0, 314.0, 305.0, 305.0, 296.0, 290.0, 283.0, 309.0, 284.0, 272.0, 265.0, 0.0, 0.0, 0.0]
header: 
seq: 21926
secs: 1603412219
nsecs: 523715525
data_max: 20.0
data_a: [inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, inf, 12.448999881744385, 4.4770002365112305, 4.513000011444092, 4.546999931335449, 4.571000099182129, 4.61299991607666, 4.64900016784668, 4.690000057220459, 4.711999893188477, 4.763999938964844, 0.003000000026077032, 0.001000000026077032, 0.003000000026077032, 0.003000000026077032]
data_b: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 325.0, 321.0, 475.0, 640.0, 375.0, 339.0, 322.0, 309.0, 304.0, 304.0, 382.0, 336.0, 0.0, 0.0, 0.0, 307.0, 292.0, 0.0, 321.0, 388.0, 341.0, 0.0, 0.0, 0.0, 0.0, 436.0, 302.0, 303.0, 309.0, 320.0, 338.0, 345.0, 354.0, 361.0, 362.0, 397.0, 415.0, 348.0, 343.0, 340.0, 337.0, 335.0, 333.0, 325.0, 318.0, 317.0, 311.0, 310.0, 985.0, 296.0, 289.0, 281.0, 309.0, 985.0, 268.0, 0.0, 0.0, 0.0, 0.0]

顺序：seq、secc、nsecs、最小值数据-a、最大值数据-a、最小值数据-b、最大值数据-b

output.txt

21925, 1603441909, 503731023,  0.001000000026077032, 5.611999988555908, 0.0, 911.0
21926, 1603412219, 523715525, 0.001000000026077032,  12.448999881744385, 0.0, 985

def parrse_file():
    with open('log.txt', 'r') as infile: 
            for line in infile:
                chunks = line.split('header:\n')
                for chunk in chunks[1:]:           
                    lines = chunk.strip().splitlines()
                    print lines

问题是我得到了空列表。根本原因是什么？如何解析lox文件并像out.txt文件一样获取信息？

Answer 1

您正在混合几个 Python 概念。处理文件时，对文件对象进行循环与对每一行进行循环相同。下面的代码是等价的：

with open('log.txt', 'r') as infile:
    for line in infile:
        print(line)
    lines = infile.readlines()
    for line in lines:
        print(line)

这意味着，您的行变量将依次保存文件的每一行。因此，当您在 header 上拆分时，您永远不会得到预期的结果。

让我们逐行查看您的代码以了解发生了什么：

```
with open('log.txt', 'r') as infile: 
```
您创建一个上下文，其中 infile 是您的文件 log.txt
```
    for line in infile: 
```
你循环到文件对象，这实际上会循环到你的每一行文件，变量 line，将依次采用以下值：
- header:\n
- seq: 21925\n
- secs: 1603441909\n
- nsecs: 503731023\n
- data_max: 20.0\n
- ...
```
   chunks = line.split('header:\n')
```
通过将带有字符串 header:\n 的行拆分，您正在构建一个列表，基于变量 line 的值，chunks 将如下所示：
- ["header \n"]
- ["seq: 21926\n"]
- ...
```
for chunk in chunks[1:]:
```
您在这里从第二个元素 (chunks) 开始在 [1:] 列表中循环，因为 chunks 将始终是具有 1 个元素的列表，chunks[1:] 将始终是一个空列表，因此永远不会调用循环内的代码。

您想要的可能（但未优化）的实现可能是：


def parse_file():
    # store each values
    out = []
    with open('log.txt', 'r') as infile:
        # current_section
        current = []
        # loop through each line of the document
        for raw_line in infile.readlines():
            # remove end line
            line = raw_line.strip()
            if line == "header:":
                # if line is header and there is something in current, add to the output
                if len(current) > 0:
                    out.append(" ".join(current))
                # reset current
                current = []
            elif line:
                # get key and val
                line_splitted = line.split(": ")
                key = line_splitted[0]
                val = line_splitted[1]
                # Add to current
                if key in ["seq", "seqs", "nsecs"]:
                    current.append(val)
                elif key in ["data_a", "data_b"]:
                    # Parse list by removing [] and splitting on `, `
                    raw_values = val[1:-1].split(", ")
                    values = []
                    # convert value to float
                    for value in raw_values:
                        if "inf" in value:
                            # skip inf
                            continue
                        values.append(float(value))
                    # Add min max by converting to str
                    current.append(str(min(values)))
                    current.append(str(max(values)))
        # Add last value of current to out
        out.append(" ".join(current))
    return "\n".join(out)

解析混合数据日志文件

1 个答案: