存储Doc2vec gensim模型时出现Pickel错误

时间:2017-08-20 15:25:45

标签: nlp pickle gensim doc2vec

我正在尝试保存gensim Doc2vec模型。该模型使用9M文档向量和大约1M单词的词汇进行训练。但我得到了错误。 “top”表示该程序使用大约13GB的RAM。另外我认为,因为我需要在需要时为新文档重新训练模型,所以必须保存所有参数。

Traceback (most recent call last):
 File "doc_2_vec.py", line 61, in <module>

model.save("/data/model_wl_videos/model",pickle_protocol=2)
 File "/home/meghana/.local/lib/python2.7/site-packages/gensim/models/word2vec.py", line 1406, in save
super(Word2Vec, self).save(*args, **kwargs)
 File "/home/meghana.negi/.local/lib/python2.7/site-packages/gensim/utils.py", line 504, in save
pickle_protocol=pickle_protocol)
 File "/home/meghana/.local/lib/python2.7/site-packages/gensim/utils.py", line 376, in _smart_save
pickle(self, fname, protocol=pickle_protocol)
 File "/home/meghana/.local/lib/python2.7/site-packages/gensim/utils.py", line 930, in pickle
_pickle.dump(obj, fout, protocol=protocol)

的MemoryError

0 个答案:

没有答案