Question

我是Word2Vec＆＃39;在Gensim。我想为文本构建一个Word2Vec模型（从维基百科提取：机器学习）并找到最相似的单词到机器学习＆＃39;。

我目前的代码如下。

# import modules & set up logging
from gensim.models import Word2Vec

sentences = "Machine learning is the subfield of computer science that, according to Arthur Samuel, gives computers the ability to learn without being explicitly programmed.[1][2][verify] Samuel, an American pioneer in the field of computer gaming and artificial intelligence, coined the term machine learning in 1959 while at IBM. Evolved from the study of pattern recognition and computational learning theory in artificial intelligence,[3] machine learning explores the study and construction of algorithms that can learn from and make predictions on data[4] – such algorithms overcome following strictly static program instructions by making data-driven predictions or decisions,[5]:2 through building a model from sample inputs. Machine learning is employed in a range of computing tasks where designing and programming explicit algorithms with good performance is difficult or infeasible; example applications include email filtering, detection of network intruders or malicious insiders working towards a data breach,[6] optical character recognition (OCR),[7] learning to rank, and computer vision."
# train word2vec on the sentences
model = Word2Vec(sentences, min_count=1)
vocab = list(model.wv.vocab.keys())
print(vocab[:10])

然而，对于词汇我得到一个字符输出。

['M', 'a', 'c', 'h', 'i', 'n', 'e', ' ', 'l', 'r']

请使用 model.most_similar

帮助我获取most_similar_words

Answer 1

Word2Vec类期望其sentences的语料库是各个项目的可迭代来源，每个项目都是一个单词列表标记。

你提供一个字符串。如果它迭代它，它会获得单个字符。如果它然后尝试将这些单个字符解释为标记列表，它仍然只是获得单个字符 - 因此它看到的唯一“单词”是单个字符。

至少，你希望你的语料库更像这样构建：

sentences = [
    "Machine learning is the subfield of computer science that, according to Arthur Samuel, gives computers the ability to learn without being explicitly programmed.[1][2][verify] Samuel, an American pioneer in the field of computer gaming and artificial intelligence, coined the term machine learning in 1959 while at IBM. Evolved from the study of pattern recognition and computational learning theory in artificial intelligence,[3] machine learning explores the study and construction of algorithms that can learn from and make predictions on data[4] – such algorithms overcome following strictly static program instructions by making data-driven predictions or decisions,[5]:2 through building a model from sample inputs. Machine learning is employed in a range of computing tasks where designing and programming explicit algorithms with good performance is difficult or infeasible; example applications include email filtering, detection of network intruders or malicious insiders working towards a data breach,[6] optical character recognition (OCR),[7] learning to rank, and computer vision.".split(),
]

那仍然只是一个'句子'，但它会被分成空白字体。

另请注意，有用的word2vec结果需要大量不同的文本样本 - 玩具大小的示例通常不会显示word2vec以创建而闻名的单词相似性或单词相对排列。

使用model.most_similar在Gensim中使用Word2Vec

1 个答案: