如何在每个单词出现之前写出文本文件的名称?

时间:2018-11-15 21:01:48

标签: python python-3.x dictionary frequency word-frequency

我该如何在每个单词频率中写入文本文件名,以便它首先显示fileno,然后显示该文件中单词的频率。 例如: {like:['file1',2,'file2,'4']} 这里就像是两个文件都包含的单词,我想在它们的频率之前写入file1和file2。 对于任何数量的文件来说应该是通用的。

这是我的代码

$_

2 个答案:

答案 0 :(得分:1)

我知道我的代码不是很漂亮,也不完全是您想要的,但这是一个解决方案。我更喜欢使用字典而不是像['file1',2,'file2,'4']

这样的列表结构

让我们以2个文件为例:

file1.txt:

this is an example

file2.txt:

this is an example
but multi line example

这是解决方案:

from collections import Counter

filenames = ["file1.txt", "file2.txt"]

# First, find word frequencies in files
file_dict = {}
for filename in filenames:
    with open(filename) as f:
        text = f.read()
    words = text.split()

    cnt = Counter()
    for word in words:
        cnt[word] += 1
    file_dict[filename] = dict(cnt)

print("file_dict: ", file_dict)

#Then, calculate frequencies in files for each word 
word_dict = {}
for filename, words in file_dict.items():
    for word, count in words.items():
        if word not in word_dict.keys():
            word_dict[word] = {filename: count}
        else:
            if filename not in word_dict[word].keys():
                word_dict[word][filename] = count    
            else:
                word_dict[word][filename] += count


print("word_dict: ", word_dict)

输出:

file_dict:  {'file1.txt': {'this': 1, 'is': 1, 'an': 1, 'example': 1}, 'file2.txt': {'this': 1, 'is': 1, 'an': 1, 'example': 2, 'but': 1, 'multi': 1, 'line': 1}}
word_dict:  {'this': {'file1.txt': 1, 'file2.txt': 1}, 'is': {'file1.txt': 1, 'file2.txt': 1}, 'an': {'file1.txt': 1, 'file2.txt': 1}, 'example': {'file1.txt': 1, 'file2.txt': 2}, 'but': {'file2.txt': 1}, 'multi': {'file2.txt': 1}, 'line': {'file2.txt': 1}}

答案 1 :(得分:0)

这是collections.Counter的好用例;我建议为每个文件创建一个计数器。

from collections import Counter

def make_counter(filename):
    cnt = Counter()

    with open(filename) as f:
        for line in f:                # read line by line, is more performant for big files
            cnt.update(line.split())  # split line by whitespaces and updated word counts

    print(filename, cnt)
    return cnt

该功能可用于每个文件,并创建一个dict来保存所有计数器:

filename_list = ['f1.txt', 'f2.txt', 'f3.txt']
counter_dict = {                      # this will hold a counter for each file
    fn: make_counter(fn)
    for fn in filename_list}

现在set可用于获取文件中出现的所有不同单词:

all_words = set(                      # this will hold all different words that appear
    word                              # in any of the files
    for cnt in counter_dict.values()
    for word in cnt.keys())

这些行将打印每个单词以及每个文件中单词的计数:

for word in sorted(all_words):
    print(word)
    for fn in filename_list:
        print('  {}: {}'.format(fn, counter_dict[fn][word]))

显然,您可以根据自己的特定需求调整打印,但是这种方法应该可以为您提供所需的灵活性。


如果您宁愿有一个dict,且所有单词都作为键,而它们的数量作为值,则可以尝试以下操作:

all_words = {}

for fn, cnt in counter_dict.items():
    for word, n in cnt.items():
        all_words.setdefault(word, {}).setdefault(fn, 0)
        all_words[word][fn] += 0