我收到此错误消息
Traceback (most recent call last):
File "C:/Users/shengrong/Desktop/bigram", line 55, in <module>
bg = bigram(file)
File "C:/Users/shengrong/Desktop/bigram", line 43, in bigram
return tt1.perplexity(my_bigrams)
File "C:\Python27\lib\site-packages\nltk\model\ngram.py", line 217, in perplexity
return pow(2.0, self.entropy(text))
File "C:\Python27\lib\site-packages\nltk\model\ngram.py", line 205, in entropy
e += self.logprob(token, context)
File "C:\Python27\lib\site-packages\nltk\model\ngram.py", line 151, in logprob
return -log(self.prob(word, context), 2)
ValueError: math domain error
import os,csv,nltk
from nltk.model.ngram import NgramModel
from nltk.probability import LidstoneProbDist
fout = open("/Users/shengrong/Documents/personal/WN1.data.csv", "w")
outfilehandle = csv.writer(fout,
delimiter=",",
quotechar='"',
quoting=csv.QUOTE_NONNUMERIC)
localrow = []
localrow.append("File name")
localrow.append("Perplexity for unigram")
localrow.append("Perplexity for bigram")
localrow.append("Perplexity for trigram")
outfilehandle.writerow(localrow)
def bigram(file):
file_object = open(file)
ln=file_object.read()
words = nltk.word_tokenize(ln)
my_bigrams = nltk.bigrams(words)
my_trigrams = nltk.trigrams(words)
tt1=NgramModel(2, my_bigrams, estimator = None)
return tt1.perplexity(my_bigrams)
#set the path of the folder
os.chdir("/Users/shengrong/Documents/A")
s = os.getcwd()
#search files in the folder
files = os.listdir(s)
for file in files:
bg = bigram(file)
localrow= []
localrow.append(file)
localrow.append(bg)
outfilehandle.writerow(localrow)
fout.close()
如何解决这个问题?不使用循环来读取文件夹,我的代码运行良好。
谢谢你们。
答案 0 :(得分:0)
您收到的错误是数学域错误。由于它在没有循环的情况下工作,因此可能是一个或多个文件包含Math包无法处理的一些数据。
请确认您文件夹中的所有文件都包含正确格式和预期值的数据。