使用python(acora)查找包含关键字的行

时间:2018-02-12 16:56:23

标签: python search aho-corasick

我正在编写一个程序,该程序读入文本文件目录并找到重叠的字符串的特定组合(即在所有文件之间共享)。我目前的方法是从该目录中获取一个文件,解析它,构建每个字符串组合的列表,然后在其他文件中搜索此字符串组合。例如,如果我有十个文件,我会读取一个文件,解析它,存储我需要的关键字,然后在其他九个文件中搜索这个组合。我会为每个文件重复此操作(确保单个文件不会自行搜索)。为此,我正在尝试使用python的acora模块。

我到目前为止的代码是:

def match_lines(f, *keywords):
    """Taken from [https://pypi.python.org/pypi/acora/], FAQs and Recipes #3."""
    builder = AcoraBuilder('\r', '\n', *keywords)
    ac = builder.build()

    line_start = 0
    matches = False
    for kw, pos in ac.filefind(f):  # Modified from original function; search a file, not a string.
        if kw in '\r\n':
            if matches:
                yield f[line_start:pos]
                matches = False
            line_start = pos + 1
        else:
            matches = True
    if matches:
        yield f[line_start:]


def find_overlaps(f_in, fl_in, f_out):
    """f_in: input file to extract string combo from & use to search other files.
    fl_in: list of other files to search against.
    f_out: output file that'll have all lines and file names that contain the matching string combo from f_in.
    """
    string_list = build_list(f_in)  # Open the first file, read each line & build a list of tuples (string #1, string #2). The "build_list" function isn't shown in my pasted code.
    found_lines = []  # Create a list to hold all the lines (and file names, from fl_in) that are found to have the matching (string #1, string #2).
    for keywords in string_list:  # For each tuple (string #1, string #2) in the list of tuples
        for f in fl_in:  # For each file in the input file list
            for line in match_lines(f, *keywords):
                found_lines.append(line)

正如您可能知道的那样,我使用了acora网页上的函数match_lines,“常见问题和食谱”#3。我还在模式中使用它来解析文件(使用ac.filefind()),也来自网页。

代码似乎有效,但它只能让我拥有匹配字符串组合的文件名。我想要的输出是从包含我匹配的字符串组合(元组)的其他文件中写出整行。

1 个答案:

答案 0 :(得分:1)

我没有看到这会产生文件名,正如你所说的那样。

无论如何,要获取行号,只需在match_lines()中传递它们时计算它们:

line_start = 0
line_number = 0
matches = False
text = open(f, 'r').read()
for kw, pos in ac.filefind(f):  # Modified from original function; search a file, not a string.
    if kw in '\r\n':
        if matches:
            yield line_number, text[line_start:pos]
            matches = False
        line_start = pos + 1
        line_number += 1
    else:
        matches = True
if matches:
    line_number, yield text[line_start:]