Question

我有大约20000个文本文件，编号为1.txt，2.txt等等。

现在，我正在创建一个包含文件路径的字典d 5.txt，10.txt，15.txt等等。

d[value]=filepath

ex:
d[5]=d:/articles/5.txt
d[45]=d:/articles/45.txt

我有一个文本文件“temp.txt”，其中包含500字的列表

vs
mln
money

依旧......

现在对于字典“d”中的每个文本文件，我需要记录列表中所有单词的出现频率。

所以，我创建了一个d2 [word] [file] = count形式的嵌套字典（这是正确的方法吗？）

where, d2[vs][5]=number of times "vs" occurs in 5.txt

简而言之，对于每个文件，我遍历单词列表并计算其出现次数。

我如何创建d2？

我的错误代码是：

import collections, sys, os, re

sys.stdout=open('3.txt','w')
from collections import Counter
from glob import glob

folderpath='d:/individual-articles'
folderpaths='d:/individual-articles/'
counter=Counter()
filepaths = glob(os.path.join(folderpath,'*.txt'))

# returns the next word in the file
def words_generator(fileobj):
    for line in fileobj:
        for word in line.split():
            yield word

d= collections.defaultdict(list)

#to print the filenames:(creation of d)

with open('topics.txt','r') as f:
    for line in f.readlines():
        value=(line.split('~')[0])
        if int(value)%5==0:
            file=folderpaths+value+'.txt'
            d[value].append(file)

d2= collections.defaultdict(list)

for file in filepaths:
    f = open(file,"r")
    words = words_generator(f)
    for word in words:
        if  file in d[file]:
            d2[word][file]+= 1              
#i have no idea how to go further, beyond this point.

Plz帮助!!

Answer 1

这样的事情：

import os
from collections import Counter,defaultdict
d2 = defaultdict(dict)
word_list = ['vs', 'mln', 'money']
for fil in d.values():
    with open(fil[0]) as f:
       path, name = os.path.split(fil[0])
       words_c = Counter([word for line in f for word in line.split()])
       for word in word_list:
           d2[word][name] = words_c[word]

现在访问d2为：

d2['vs']['5.txt']

Answer 2

如果你可以交换字典，你可以让@Ashwini Chaudhary的答案有所简化（虽然效率可能稍低）：

from collections import Counter, defaultdict

with open('temp.txt') as f:
    word_list = set(f.read.split())

d2 = defaultdict(Counter)
for n in range(number_of_files):
    with open('{}.txt'.format(n)) as f:
        d2[fil] = Counter([word for word in f.read().split() if word in word_list])

迭代python中的嵌套字典

2 个答案: