如何根据Python中的前一行/字符串获取字符串中字符的位置?

时间:2020-06-24 16:12:32

标签: python pandas parsing text-parsing

我必须解析一个文件,该文件包含我需要放入/排序的数据到pandas数据框中。以下是我解析的一部分文件的示例:

TEST#          RESULT    UNITS          LOWER          UPPER ALARM TEST NAME
------ -------------- -------- -------------- -------------- ----- ----------
1.0              1234                                              TESTNAME1
1.1           H647333                                              TESTNAME2
1.2                30        C                                     TEMPTOTAL
1.3                 1      cnt                                     CEREAL
1.4            364003      cnt                                     POINTNUM
1.5          20200505      cnt                                     Date
1.6            174143      cnt                                     Time
1.7     2.020051e+007      cnt                                     DateTime
1.8               123      cnt                                     SMT
1.9             23.16        C                                     TEMP1
1.10            23.55        C                           123       TEMP2
1.11            22.88        C            -23                      TEMP3
1.12            22.86        C                                     TEMP4
1.13            1.406    Meter         -1.450          1.500       DIST1
1.14            0.718    Meter         -0.800          0.350  FAIL DIST2

我的问题是: 我如何解释没有下限但没有上限,或者有上限但没有下限?

注意:我的实际文本文件没有这种情况,但是我的application \ project调用了可能发生这种情况的实例。

我如何检查每行如下:

 line = file_object.readline()
 while line.strip():
    # extract data from line and format all info in one list
    xline = line.strip().split()

    # the length of the info list of the line read
    # is correlated to the data

    if len(xline) == 3:
        number = xline[0]
        results = xline[1]
        testname = xline[2]
        units = None
        lower = None
        upper = None
        # alarm = None

    elif len(xline) == 4:
        number = xline[0]
        results = xline[1]
        units = xline[2]
        testname = xline[3]
        lower = None
        upper = None
        # alarm = None

    elif len(xline) == 6:
        number = xline[0]
        results = xline[1]
        units = xline[2]
        lower = xline[3]
        upper = xline[4]
        testname = xline[5]
        # alarm = None

    elif len(xline) == 7:
        number = xline[0]
        results = xline[1]
        units = xline[2]
        lower = xline[3]
        upper = xline[4]
        # alarm = xline[5]
        testname = xline[6]

    # create a dictionary containing this row of data
    row = {
        'Test #': number,
        'Result': results,
        'Units': units,
        'Lower Limit': lower,
        'Upper Limit': upper,
        # 'Alarm': alarm,
        'Test Name': testname,
    }
    data.append(row)
    line = file_object.readline()

我的想法是,我将读取的每一行数据与“ TEST#结果单元较低的上层警报测试名称”行标题位置进行比较,但是我不知道该怎么做。如果有人能指出我一个可行的方向,那就太好了!

编辑:文件不仅仅具有上面显示的表格格式。我的文件在文件开始处有一堆交错的块文本。以及多个“表”,它们之间有交错的块文本。

2 个答案:

答案 0 :(得分:2)

您可以使用pd.read_fwf

df = pd.read_fwf(inputtxt,'infer')

输出:

     TEST#          RESULT     UNITS           LOWER           UPPER  ALARM   TEST NAME
0   ------  --------------  --------  --------------  --------------  -----  ----------
1      1.0            1234       NaN             NaN             NaN    NaN   TESTNAME1
2      1.1         H647333       NaN             NaN             NaN    NaN   TESTNAME2
3      1.2              30         C             NaN             NaN    NaN   TEMPTOTAL
4      1.3               1       cnt             NaN             NaN    NaN      CEREAL
5      1.4          364003       cnt             NaN             NaN    NaN    POINTNUM
6      1.5        20200505       cnt             NaN             NaN    NaN        Date
7      1.6          174143       cnt             NaN             NaN    NaN        Time
8      1.7   2.020051e+007       cnt             NaN             NaN    NaN    DateTime
9      1.8             123       cnt             NaN             NaN    NaN         SMT
10     1.9           23.16         C             NaN             NaN    NaN       TEMP1
11    1.10           23.55         C             NaN             123    NaN       TEMP2
12    1.11           22.88         C             -23             NaN    NaN       TEMP3
13    1.12           22.86         C             NaN             NaN    NaN       TEMP4
14    1.13           1.406     Meter          -1.450           1.500    NaN       DIST1
15    1.14           0.718     Meter          -0.800           0.350   FAIL       DIST2

而且,您可以删除索引0以获取破折号:

df = df.drop(0)

输出:

   TEST#         RESULT  UNITS   LOWER  UPPER ALARM  TEST NAME
1    1.0           1234    NaN     NaN    NaN   NaN  TESTNAME1
2    1.1        H647333    NaN     NaN    NaN   NaN  TESTNAME2
3    1.2             30      C     NaN    NaN   NaN  TEMPTOTAL
4    1.3              1    cnt     NaN    NaN   NaN     CEREAL
5    1.4         364003    cnt     NaN    NaN   NaN   POINTNUM
6    1.5       20200505    cnt     NaN    NaN   NaN       Date
7    1.6         174143    cnt     NaN    NaN   NaN       Time
8    1.7  2.020051e+007    cnt     NaN    NaN   NaN   DateTime
9    1.8            123    cnt     NaN    NaN   NaN        SMT
10   1.9          23.16      C     NaN    NaN   NaN      TEMP1
11  1.10          23.55      C     NaN    123   NaN      TEMP2
12  1.11          22.88      C     -23    NaN   NaN      TEMP3
13  1.12          22.86      C     NaN    NaN   NaN      TEMP4
14  1.13          1.406  Meter  -1.450  1.500   NaN      DIST1
15  1.14          0.718  Meter  -0.800  0.350  FAIL      DIST2

答案 1 :(得分:0)

非熊猫解决方案,可以从标头推断字段宽度,但使用熊猫?:

import re

with open('table.txt') as fin:
    next(fin) # skip headers
    # capture start/end of each set of dashed lines to get field widths
    spans = [m.span() for m in re.finditer(r'-+',next(fin))]
    for line in fin:
        # break lines on the field widths and strip leading/trailing white sapce
        column = [line[start:end].strip() for start,end in spans]
        print(column)

输出:

['1.0', '1234', '', '', '', '', 'TESTNAME1']
['1.1', 'H647333', '', '', '', '', 'TESTNAME2']
['1.2', '30', 'C', '', '', '', 'TEMPTOTAL']
['1.3', '1', 'cnt', '', '', '', 'CEREAL']
['1.4', '364003', 'cnt', '', '', '', 'POINTNUM']
['1.5', '20200505', 'cnt', '', '', '', 'Date']
['1.6', '174143', 'cnt', '', '', '', 'Time']
['1.7', '2.020051e+007', 'cnt', '', '', '', 'DateTime']
['1.8', '123', 'cnt', '', '', '', 'SMT']
['1.9', '23.16', 'C', '', '', '', 'TEMP1']
['1.10', '23.55', 'C', '', '123', '', 'TEMP2']
['1.11', '22.88', 'C', '-23', '', '', 'TEMP3']
['1.12', '22.86', 'C', '', '', '', 'TEMP4']
['1.13', '1.406', 'Meter', '-1.450', '1.500', '', 'DIST1']
['1.14', '0.718', 'Meter', '-0.800', '0.350', 'FAIL', 'DIST2']