Python NLTK多线程

时间:2015-10-12 08:30:29

标签: python multithreading lazy-loading nltk

我正在编写一种算法,用于识别给定文本中的句子,将每个句子分成单词&经过一些验证后返回这些单词。

我希望在多线程的帮助下实现相同的功能。

我正在调用我的函数来处理threading.thread()中为其抛出错误的每个句子:

AttributeError: 'WordListCorpusReader' object has no attribute '_LazyCorpusLoader__args'  

然而,很少有博客建议使用" wn.ensure_loaded()"功能。

但是python抛出一个错误,说ensure_loaded()没有定义。

有人可以帮我解决这个问题。

编辑:

text = "This is my sample text. I want to break it into sentences"  
sentences=(re.split(r"(?<!\w\.\w.)(?<![A-Z][a-z]\.)(?<=\.|\?)\s",text))  
wn.ensure_loaded()      
co = CoOccurence()     #CoOccurence is a class in my package which will work on the sentences  
for sentence in sentences:  
    t = Thread(target=co.__prepareHash__, args=(sentence,)) #co.__prepareHash__   is an other method which works around the sentences  
    threads.append(t)  
    t.start()  

flag =1  
while (flag):  
    flag = __isThreadAlive__()  

这引发了一个错误:Attribute error: 'WordNetCorpusReader' object has no attribute '_LazyCorpusLoader__args'
当我尝试检查wn.ensure_loaded()时,它会抛出AttributeError: 'module' object has no attribute 'ensure_loaded'

与ref。致:this SO question

谢谢

0 个答案:

没有答案