sklearn.metrics.pairwise的两个稀疏矩阵的余弦相似性得到MemoryError

时间:2020-02-04 06:38:10

标签: python sparse-matrix matrix-multiplication

我正在尝试建立一个推荐系统,该系统需要查找所有电影之间的余弦相似度。

hybrid_recommender.cosine_sim.append(cosine_similarity(count_matrix,count_matrix))

Traceback (most recent call last):

  File "<ipython-input-10-dbe17634b4d6>", line 1, in <module>
    hybrid_recommender.cosine_sim.append(cosine_similarity(count_matrix, count_matrix))

  File "C:\Users\Niranjan Kumar\Anaconda3\lib\site-packages\sklearn\metrics\pairwise.py", line 1176, in cosine_similarity
    dense_output=dense_output)

  File "C:\Users\Niranjan Kumar\Anaconda3\lib\site-packages\sklearn\utils\extmath.py", line 155, in safe_sparse_dot
    return ret.toarray()

  File "C:\Users\Niranjan Kumar\Anaconda3\lib\site-packages\scipy\sparse\compressed.py", line 962, in toarray
    out = self._process_toarray_args(order, out)

  File "C:\Users\Niranjan Kumar\Anaconda3\lib\site-packages\scipy\sparse\base.py", line 1187, in _process_toarray_args
    return np.zeros(self.shape, dtype=self.dtype, order=order)
MemoryError

矩阵的大小有点大133594x43714

count_matrix
Out[17]: 
<133594x43714 sparse matrix of type '<class 'numpy.int64'>'
    with 830942 stored elements in Compressed Sparse Row format>

有什么方法可以实现这种矩阵乘法,我正在使用带i7和16 GB内存的Windows 10笔记本电脑。

0 个答案:

没有答案
相关问题