MapReduce用于计算文本文件中单词中辅音数量的频率

时间:2014-06-17 13:56:23

标签: python python-2.7 amazon-web-services mapreduce

我需要一些Python代码的帮助来计算一个单词中辅音的频率。请考虑以下示例输入:

"There is no new thing under the sun."

然后所需的输出将是:

1 : 2
2 : 3
3 : 2
4 : 1

有2个单词,1个辅音,3个单词,2个辅音,2个单词,3个辅音,1个单词,4个辅音。

以下代码执行类似的工作,但它不是辅音,而是计算文本文件中整个单词的频率。我知道只有一点点变化会更深入地循环到这个词(我想)。

def freqCounter(file1, file2):
    freq_dict = {}
    dict_static = {2:0, 3:0, 5:0}
    # get rid of punctuation
    punctuation = re.compile(r'[.?!,"\':;]') # use re.compile() function to convert string into a RegexObject. 
    try:
        with open(file1, "r") as infile, open(file2, "r") as infile2: # open two files at once
            text1 = infile.read()   # read the file
            text2 = infile2.read()
            joined = " ".join((text1, text2)) 
            for word in joined.lower().split(): 
                #remove punctuation mark
                word = punctuation.sub("", word)
                #print word
                l = len(word) # assign l tp be the word's length
                # if corresponding word's length not found in dict
                if l not in freq_dict:
                    freq_dict[l] = 0 # assign the dict key (the length of word) to value = 0
                freq_dict[l] += 1 # otherwise, increase the value by 1
    except IOError as e:     # exception catch for error while reading the file
        print 'Operation failed: %s' % e.strerror
    return freq_dict # return the dictionary

非常感谢任何帮助!

3 个答案:

答案 0 :(得分:1)

我会尝试一种更简单的方法:

from collections import Counter
words = 'There is no new thing under the sun.'
words = words.replace('a', '').replace('e', '').replace('i', '').replace('o', '').replace('u', '')  # you are welcome to replace this with a smart regex

# Now words have no more vowels i.e. only consonants 
word_lengths = map(len, words.split(' '))
c = Counter(word_lengths)

freq_dict = dict(Counter(c))

答案 1 :(得分:1)

一个简单的解决方案

def freqCounter(_str):
    _txt=_str.split()
    freq_dict={}
    for word in _txt:
        c=0
        for letter in word:
           if letter not in "aeiou.,:;!?[]\"`()'":
               c+=1
        freq_dict[c]=freq_dict.get(c,0)+ 1
    return freq_dict

txt = "There is no new thing under the sun."
table=freqCounter(txt)
for k in table:
    print( k, ":", table[k])

答案 2 :(得分:0)

这个怎么样?

with open('conts.txt', 'w') as fh:
    fh.write('oh my god becky look at her butt it is soooo big')

consonants = "bcdfghjklmnpqrstvwxyz"
def count_cons(_file):
    results = {}
    with open(_file, 'r') as fh:
        for line in fh:
            for word in line.split(' '):
                conts = sum([1 if letter in consonants else 0 for letter in word])
                if conts in results:
                    results[conts] += 1
                else:
                    results[conts] = 1
    return results

print count_cons('conts.txt')

错过了结果

{1: 5, 2: 5, 3: 1, 4: 1}
[Finished in 0.0s]