Question

我使用struct来格式化固定分隔的txt文件。以下是前两行：

    Sat Jan  3 18:15:05 2009    62e907b15cbf27d5425399ebf6f0fb50ebb88f18    4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b +              50.00000000
    Fri Jan  9 02:54:25 2009    119b098e2e980a229e139a9ed01a469e518e6f26    0e3e2357e806b6cdb1f70b54c3a3a17b6714ee1f0e68bebb44a74b1efd512098 +              50.00000000

使用以下代码将其转换为csv：

import csv
import struct

fieldwidths = (-4, 24, -4, 40,-4,64,-1,1,25)  # negative widths represent ignored padding fields
fmtstring = ' '.join('{}{}'.format(abs(fw), 'x' if fw < 0 else 's')
                        for fw in fieldwidths)
fieldstruct = struct.Struct(fmtstring)
parse = fieldstruct.unpack_from

c = csv.writer(open("/home/ulrich/Desktop/disertation/sample_parsed_blch1.csv", "wb"))
with open('/home/ulrich/Desktop/disertation/sample_parsed_blch2.txt') as f:
    for line in f:
        fields = parse(line)
        c.writerow(fields)

它生成csv时工作正常，但我仍然收到此错误消息：

error: unpack_from requires a buffer of at least 167 bytes

Answer 1

这种错误可能是由文件末尾的额外字符引起的。已知Windows世界的一些编辑器会在文本文件的末尾添加Ctrl-Z。这是MS / DOS希望与CP / M保持兼容的时间的回忆。

您可以通过跳过较短的行来轻松摆脱它：

for line in f:
    if len(line) >= minsize # 100
        fields = parse(line)
        c.writerow(fields)

Answer 2

如果您重复该文件，line将contian尾随换行符。你需要删除它：

....
for line in f:
    fields = parse(line.rstrip('\r\n'))
    c.writerow(fields)

Answer 3

我最后添加了一个带正则表达式的条件，以便只解析具有适当格式的行

import csv
import struct
import re
pattern = re.compile("\s{4}\w{3}\s{1}.+")


fieldwidths = (-4, 24, -4, 40,-4,64,-1,1,25)  # negative widths represent ignored padding fields
fmtstring = ' '.join('{}{}'.format(abs(fw), 'x' if fw < 0 else 's')
                        for fw in fieldwidths)
fieldstruct = struct.Struct(fmtstring)
parse = fieldstruct.unpack_from

c = csv.writer(open("/media/ulrich/FC9A-C444/all_tx.csv", "wb"))
with open('/media/ulrich/FC9A-C444/all_tx.txt') as f:
    for line in f:
        if pattern.match(line):
            fields = parse(line)
            c.writerow(fields)

错误：unpack_from需要缓冲区

3 个答案: