如何实现pythonic行连续阅读

时间:2019-05-30 17:18:30

标签: python regex python-3.x file

我正在尝试实现一个python脚本来读取和提取ASCII文本文件中的行。这似乎是一件很容易的事情,但是我最终遇到了一个我无法解决的问题。 我尝试读取的文件包含测试和某些行 以service开头。该模式可以是大写或小写字母以及数字位数,并且*tr999的出现是可选的。星号也可以在之前和之后。该信号关键字后跟数字,即int或folat。为了捕获信号,我使用了python regexp表达式

*

文本文件如下

re.search("[*]{0,1}[Tt][Rr][0-9]{1,5}[*]{0,1}",line)

我编写的代码无法捕获最后一种情况。缺少连续行信号tr10* 1 2 3 22 1 1 13 12 33 33 33 *Tr20 12 22 -1 2 2 2 5 5 5 6 6 6 77 Tr20 1 1 1 & 2 0 0 1 1 1 2 2 2 c that is a comment and below is the problem case '&' is missing *tr22221 2 2 2 1 1 1 2 2 2 的位置。使用&来续行是可选的,并且在续行开始时可以用许多空格代替。

我写的代码是

&

输出为:

import sys

fp=open(sys.argv[1],'r')
import re 

# get the integers only
def loop_conv(string):
        conv=[]
        for i in string.split(" "):
            try:
                conv.append(float(i))
            except ValueError:
                pass
        return conv

# extract the information
def extract_trans_card(line,fp):
            extracted=False
            if len(line)>2 and not re.search("[cC]",line.split()[0]) and re.search("[*]{0,1}[Tt][Rr][0-9]{1,5}[*]{0,1}",line) :
                extracted=True
                trans_card=[]
                trans_card.append(line.split()[0])
                line_old=line
   # this part here is because after the read signal,
   # data to be extracted might be on the same line             
                for val in loop_conv(line):
                        trans_card.append(val)
# this part here fails. I am not able to catch the case '&' missing.
# i tried to peek the next line with seek() but it i got a system error. 
# the idea is to loop until i have a continue line case  
                while (re.search("^(\s){5,60}",line) or re.search("[&$]",line_old)) and len(trans_card) <13:

                    line=fp.readline()
                    for val in loop_conv(line):
                        trans_card.append(val)
                    line_old=line


                #print('M',trans_card)
                print('value',trans_card)
                trans_card=[]
            return extracted 



# read the file with a loop
for line in fp:
        if not extract_trans_card(line,fp) :
            print(line,end='')  

最后一行是问题。由于value ['tr10*', 1.0, 2.0, 3.0, 22.0, 1.0, 1.0, 13.0, 12.0, 33.0, 33.0, 33.0] value ['*Tr20', 12.0, 22.0, -1.0, 2.0, 2.0, 2.0, 5.0, 5.0, 5.0, 6.0, 6.0, 6.0, 77.0] value ['Tr20', 1.0, 1.0, 1.0, 2.0, 0.0, 0.0, 1.0, 1.0, 1.0, 2.0, 2.0, 2.0] c that is a comment and below is the problem case '&' is missing value ['*tr22221', 2.0, 2.0, 2.0] 1 1 1 2 2 2 1 1 1被忽略并仅被回显。 这个问题看起来类似于python如何继续行。通过空格或使用2 2 2。 希望有人能帮助我们解决这个问题,并指出解决此问题的正确方法

2 个答案:

答案 0 :(得分:1)

代码工作流程的问题是,当连续行信号为可选时,将很难检测到与 current trans_card 相关的最后一行下一个 trans_card

由于可以使用re.search(r"[*]?[Tt][Rr][0-9]{1,5}[*]?"找到转帐卡的开头(标题),因此只要在 header 模式为检测到。

下面是示例代码,我从您的代码逻辑中粗略复制了这些代码,并将生成的trans_card保存到列表列表中:

import sys
import re

# get the floats only from line, copied from your code
def loop_conv(string):
    conv=[]
    for i in string.split(" "):
      try:
        conv.append(float(i))
      except ValueError:
        pass
    return conv

# set previous trans_card with non-EMPTY vals list
def set_prev_trans_card(card, vals):
    if len(vals):
        card.append(vals)
        #print ('value: {}'.format(vals))

# below new code logic:
with open(sys.argv[1], 'r') as fp:
    trans_card = []

    # a list to save items retrieved from lines associated with the same trans_card
    values = []

    # set up a flag to identify header
    is_header = 0

    for line in fp:
        # if line is a comment, then skip it 
        if re.search("[cC]",line.split()[0]):
            #print(line, end='')
            continue

        # if line is a header, append the existing values[] (from the previous trans_card) 
        # list to trans_card[] and then reset values[]
        if len(line)>2 and re.search(r"[*]?[Tt][Rr][0-9]{1,5}[*]?", line):
            # append values[] to trans_card
            set_prev_trans_card(trans_card, values)

            # reset values[] to the first \S+ on the header 
            values = [ line.split()[0] ]

            # set is_header flag to 1
            is_header = 1

        # if line ends with &\n, then concatenate the next lines
        while line.endswith('&\n'):
            line += ' ' + fp.readline()

        # add all numbers(floats) from header or lines starts with 5-60 white-spaces into the values[] list, and reset is_header flag to 0
        if is_header or re.search("^(\s){5,60}",line):
            values.extend(loop_conv(line))
            is_header = 0

    # append the last values[] to trans_card
    set_prev_trans_card(trans_card, values)

for v in trans_card:
    print ('value: {}'.format(v))

输出为:

value: ['tr10*', 1.0, 2.0, 3.0, 22.0, 1.0, 1.0, 13.0, 12.0, 33.0, 33.0, 33.0]
value: ['*Tr20', 12.0, 22.0, -1.0, 2.0, 2.0, 2.0, 5.0, 5.0, 5.0, 6.0, 6.0, 6.0, 77.0]
value: ['Tr20', 1.0, 1.0, 1.0, 2.0, 0.0, 0.0, 1.0, 1.0, 1.0, 2.0, 2.0, 2.0]
value: ['*tr22221', 2.0, 2.0, 2.0, 1.0, 1.0, 1.0, 2.0, 2.0, 2.0]

注意:我跳过了代码中len(trans_card) <13的条件,以为它只是用来防止无限的while循环。如果没有,应该很容易将其添加到上面的示例代码中。

顺便说一句。您可能想在注释和标题的模式中添加^,以便它们仅匹配字符串的开头,而不是搜索字符串中的任何位置。

答案 1 :(得分:0)

这是一种处理文件的Python方式(实际上,在 next()项目返回字符串的情况下,任何可迭代的东西都返回字符串,可能以换行符结尾或不以换行符结尾),可以通过当前“记录”的最后一列中的“&”(Python实际上使用“ \”),或者下一个“记录”中的空格字符:

import re


def read_lines_with_continue(iter):
    """This function is passed an interator where each iteration returns the next line.
       This function processes logical continuations consisting of lines that end with '&' or lines
       that begin a space."""

    next_line = ''
    saw_continue = True
    for line in iter:
        # get rid of any trailing '&'
        edited_line = re.sub(r'&$', '', line)
        if saw_continue:
            next_line += edited_line
            saw_continue = False
        elif line[0] == ' ':
            next_line += edited_line
        elif next_line != '':
            yield next_line
            next_line = edited_line
        if line != edited_line:
            saw_continue = True
    if next_line != '':
        yield next_line


lines = [
    '1abc',
    '2def&',
    'ghi',
    ' xyz',
    ' ver&',
    'jkl',
    '3aaa',
    '4xxx',
    ' yyy'
]


# instead of passing a list, you could also pass a file
for l in read_lines_with_continue(lines):
    print(l)

1abc
2defghi xyz verjkl
3aaa
4xxx yyy