KeyError(“单词'%s'不在词汇表中”%word)

时间:2019-09-19 06:32:08

标签: python machine-learning deep-learning nlp word2vec

将我的预测标签从图像转换为列表all_tag,然后将其拆分,最后存储到word_list中,其中所有标签都存储在类似结构的句子中。

我要做的就是使用Google的Word2Vec预训练模型(https://mccormickml.com/2016/04/12/googles-pretrained-word2vec-model-in-python/)生成并打印我的预测标签的所有Word2Vec值。导入并映射了模型的预训练权重,但出现错误

  

KeyError:“单词'['cliff“不在词汇表中

但是,词典中有单词“ cliff”。任何见识将不胜感激。 请检查以下代码段以供参考。

execution_path = os.getcwd()
TEST_PATH = '/home/guest/Documents/Aikomi'


prediction = ImagePrediction()
prediction.setModelTypeAsDenseNet()
prediction.setModelPath(os.path.join(execution_path, "/home/guest/Documents/Test1/ImageAI-master/imageai/Prediction/Weights/DenseNet.h5"))
prediction.loadModel()

pred_array = np.empty((0,6), dtype=object)

predictions, probabilities = prediction.predictImage(os.path.join(execution_path, "1.jpg"), result_count=5)

for img in os.listdir(TEST_PATH):
    if img.endswith('.jpg'):
        image = Image.open(os.path.join(TEST_PATH, img))
        image = image.convert("RGB")
        image = np.array(image, dtype=np.uint8)
        predictions, probabilities = prediction.predictImage(os.path.join(TEST_PATH, img), result_count=5)
        temprow = np.zeros((1,pred_array.shape[1]),dtype=object)
        temprow[0,0] = img
        for i in range(len(predictions)):
            temprow[0,i+1] = predictions[i]
        pred_array = np.append(pred_array, temprow, axis=0)


all_tags = list(pred_array[:,1:].reshape(1,-1))
_in_sent = ' '.join(list(map(str, all_tags)))


import gensim
from gensim.models import Word2Vec
from nltk.tokenize import sent_tokenize, word_tokenize
import re
import random
import nltk
nltk.download('punkt')


word_list = _in_sent.split() 

from gensim.corpora.dictionary import Dictionary

# be sure to split sentence before feed into Dictionary
word_list_2 = [d.split() for d in word_list]
dictionary = Dictionary(word_list_2)
print("\n", dictionary, "\n")

corpus_bow = [dictionary.doc2bow(doc) for doc in word_list_2]

model = Word2Vec(word_list_2, min_count= 1)
model = gensim.models.KeyedVectors.load_word2vec_format('/home/guest/Downloads/Google.bin', binary=True)

print(*map(model.most_similar, word_list))

1 个答案:

答案 0 :(得分:1)

答案就在那里,您已经很清楚地打印了

KeyError(“word '%s' not in vocabulary” % word)

,错误是

  

KeyError:“单词'['cliff“不在词汇表中

由于可变字的内容应在'和'之间 因此,单词variable具有字符串['cliff'而不是字符串cliff

从文本中删除标点符号,例如'和[]等。