Question

我有一个文本文件，每行包含一些单词。现在给出一组查询词，我必须找到文件中查询词共同出现的行数。即包含两个查询字的行数，包含3个查询字的行数等。

我尝试使用以下代码：请注意，rest（list，word）会从“list”中删除“word”并返回更新后的列表。 linecount是raw中的行数。

raw=open("raw_dataset_1","r")
queryfile=open("queries","r")
query=queryfile.readline().split()
query_size=len(query)
two=0
three=0
four=0

while linecount>0:
    line=raw.readline().split()
    if query_size>=2:
        for word1 in query:
            beta=rest(query,word1)
            for word2 in beta:
                if (word1 in line) and (word2 in line):
                    two+=1
                    print line
    if (query_size>=3):
        for word3 in query:
            beta=rest(query,word3)
            for word4 in beta:
                gama=rest(beta,word4)
                for word5 in gama:
                    if (((word3 in line) and (word4 in line)) and (word5 in line)):
                        three+=1
                        print line
    linecount-=1

print two
print three

它有效，虽然有冗余，我可以将“两个”除以2得到所需数量）有没有更好的方法呢？

Answer 1

我会采取更一般的方法。假设query是您的查询字词列表，raw_dataset_1是您正在分析的文件的名称，我会这样做：

# list containing the number of lines with 0,1,2,3... occurrances of query words.
wordcount = [0,0,0,0,0]    
for line in file("raw_dataset_1").readlines():
    # loop over each query word, see if it occurs in the given line, and just count them. 
    # The bracket inside will create a list of elements (query_word) from your query word list (query)
    # but add only those words which occur in the line (if  query_word in line). [See list comprehension]
    # E.g. if your line contain three query words those three will be in the list.
    # You are not interested in what those words are, so you just take the length of the list (len). 
    # Finally, number_query_words_found is the number of query words present in the current line of text. 
    number_query_words_found = len([query_word for query_word in query if query_word in line])  
    if number_query_words_found<5:
        # increase the line-number by one. The index corresponds to the number of query-words present
        wordcount[number_query_words_found] += 1

print "Number of lines with 2 query words: ", wordcount[2]
print "Number of lines with 3 query words: ", wordcount[3]

此代码未经过测试且可以进行优化。该文件将被完全读取（对于较大的文件来说效率低）并且列表wordcount是静态的，应该动态完成（以允许任何单词出现。但是这样的事情应该有效，除了我误解了你的问题。对于列表理解参见例如here。

Answer 2

我会使用套装：

raw=open("raw_dataset_1","r")
queryfile=open("queries","r")
query_line = queryfile.readline()
query_words = query_line.split()
query_set = set(query_words)
query_size = len(query_set)  # Note that this isn't actually used below

for line in raw: # Iterating over a file gives you one line at a time
    words = line.strip().split()
    word_set = set(words)
    common_set = query_set.intersection(word_set)
    if len(common_set) == 2:
        two += 1
    elif len(common_set) == 3:
        three += 1
    elif len(common_set) == 4:
        four += 1

当然，您可能希望将行保存到结果文件或其他任何内容，而不仅仅计算出现次数。但是这应该给你一般的想法：使用集合将极大地简化你的逻辑。

找到一行中给定单词的常见事件

2 个答案: