Question

我在文本文件中有表格数据，所以我试图使用python获取数据，但我找不到每个列之间的分隔符。请帮帮我。提前谢谢。

数据可能如下所示：

Column1           Column2         Column3            Column4
----------------------------------------------------------------------------
apple fruits      banana fruits     orange fruits    grapes fruits
mango fruits      pineapple fruits                   blackberry fruits
                  blueberry fruits  currant fruits   papaya fruits
chico fruits                        peach fruits     pear fruits

我的预期结果是字典格式。

Answer 1

我正在假设数据在每条记录的相同列中对齐。

我将标题行和典型行放在两个distict变量中，您将从文件中读取它们

>>> a = 'Column1           Column2             Column3             Column4'
>>> b = 'apple fruits      banana fruits       orange fruits       grapes fruits'

i是标题中的索引列表，最初为空，inside表示我们位于列名称内

>>> i = []
>>> inside = False

我们统计字符并检查我们是否在列名的开头

>>> for n, c in enumerate(a):
...     if c == ' ':
...         inside = False
...         continue
...     if not inside:
...         inside = True
...         i.append(n)
>>> i
[0, 18, 38, 58]

我们有列的开头的索引，下一个的开头是，在切片表示法中，也是当前的结尾 - 我们只需要最后一列的结尾但是使用切片表示法我们可以使用值None

>>> [b[j:k].rstrip() for j, k in zip(i,i[1:]+[None])]
['apple fruits', 'banana fruits', 'orange fruits', 'grapes fruits']

当然，您必须对输入文件中的每个数据行应用相同的索引技巧。

P.S。：您可能希望使用

中的itertools.zip_longest方法

[... for j, k in itertools.zip_longest(i, i[1:])]

您可能希望缓存生成器以避免为每个数据行实例化

cached_indices = list(itertools.zip_longest(i, i[1:]))
for line in data:
    c1, c2, c3, c4 = [... for i, j in cached_indices]

我尝试在下面的评论中实施我的建议，这是我的最大努力......

$ cat fetch.py
from itertools import count  # this import is necessary
from io import StringIO      # this one is needed to simulate an open file

# Your data, notice that some field in the last two lines is misaligned
data = '''\
Column1           Column2           Column3          Column4
----------------------------------------------------------------------------
apple fruits      banana fruits     orange fruits    grapes fruits
mango fruits      pineapple fruits                   blackberry fruits
                   blueberry fruits currant fruits   papaya fruits
chico fruits                        peach fruits    pear fruits
'''

f = StringIO(data) # you may have something like
                   # f = open('fruitfile.fixed')

# read the header line and skip a line                   
header = next(f).rstrip()
next(f) # skip a line

# a compact way of finding the starts of the columns
indices = [i for i, c0, c1 in zip(count(), ' '+header, header)
           if c0==' ' and c1!=' ']
# We are going to reuse zip(indices, indices[1:]+[None]), so we cache it
ranges = list(zip(indices, indices[1:]+[None]))

# we are ready for a loop on the lines of the file
for nl, line in enumerate(f, 3):
    if line == '\n': continue # don't process blank lines
    # extract the _raw_ fields from a line
    fields = [line[i:j] for i, j in ranges]
    # check that a non-all-blanks field does not start with a blank,
    # check that a field does not terminate wit anything but a space
    # or a newline
    if any((f[0]==' ' and f.rstrip()) or f[-1] not in ' \n' for f in fields):
        # signal the possibility of a misalignment
        print('Possible misalignment in line n.%d:'%nl)
        print('\t|'+header)
        print('\t|'+line.rstrip())
    # the else body is executed if all the fields are OK
    # what I do with the fields is just a possibility
    else:
        print('Data Line n.%d:'%nl)
        fields = [field.rstrip() for field in fields]
        for nf, field in enumerate(fields, 1):
            print('\tField n.%d:\t%r'%(nf, field))

$ python3 fetch.py 
Data Line n.3:
        Field n.1:      'apple fruits'
        Field n.2:      'banana fruits'
        Field n.3:      'orange fruits'
        Field n.4:      'grapes fruits'
Data Line n.4:
        Field n.1:      'mango fruits'
        Field n.2:      'pineapple fruits'
        Field n.3:      ''
        Field n.4:      'blackberry fruits'
Possible misalignment in line n.5:
        |Column1           Column2           Column3          Column4
        |                   blueberry fruits currant fruits   papaya fruits
Possible misalignment in line n.6:
        |Column1           Column2           Column3          Column4
        |chico fruits                        peach fruits    pear fruits
$

Answer 2

[0, 18, 38, 58]的魔力，列的起始位置，也在我的回答中起作用，但它基于numpy.genfromtxt()

from pathlib import Path
import pandas as pd
import numpy as np

# replicate the file
doc = """Column1           Column2             Column3             Column4
----------------------------------------------------------------------------
apple fruits      banana fruits       orange fruits       grapes fruits
mango fruits      pineapple fruits                        blackberry fruits
                  blueberry fruits    currant fruits      papaya fruits
chico fruits                          peach fruits        pear fruits"""

Path('temp.txt').write_text(doc)

# read the file    
lines = Path('temp.txt').read_text().split('\n')

# play with header to find the column widths
header = lines[0]
length = max([len(line) for line in lines])
starts = [i for i, char in enumerate(header) if char=='C'] + [length]
widths = [x-prev for x, prev in zip(starts[1:], starts[:-1])] 
assert sum(widths) == length
data = np.genfromtxt('temp.txt', dtype=None, delimiter=widths, autostrip=True,
                     encoding='utf-8')

# make pandas dataframe 
colnames = [x for x in header.split(' ') if x]
df = pd.DataFrame(data[2:], columns=colnames)

# check it is what we wanted
assert df.to_csv(index=False) == \
"""Column1,Column2,Column3,Column4
apple fruits,banana fruits,orange fruits,grapes fruits
mango fruits,pineapple fruits,,blackberry fruits
,blueberry fruits,currant fruits,papaya fruits
chico fruits,,peach fruits,pear fruits
"""

如何使用python

2 个答案: