Question

我正在尝试使用大阵列进行点积运算

L_mat=csr_matrix(L_mat)
# L_mat is (24, 1226880)

L_mat_t=csr_matrix(L_mat_t)
# L_mat_t is (1226880, 24)

# Following is possible?
LT_dot_L=L_mat_t.dot(L_mat)
# I'm expecting (1226880, 1226880) but when I did this, 
# I got MemoryError

当我执行较小的数组时，我遇到了相同的MemoryError

(523776, 24) dot (24, 523776)

可以执行

(24, 523776) dot (523776, 24) = (24, 24)

如何使用csr_matrix或其他方式执行大型阵列点积？

Answer 1

我不会尝试发生内存错误，但考虑进行此测试

In [300]: from scipy import sparse
In [301]: M = sparse.random(10,10000, format='csr')
In [302]: M
Out[302]: 
<10x10000 sparse matrix of type '<class 'numpy.float64'>'
    with 1000 stored elements in Compressed Sparse Row format>
In [303]: M.dot(M.T)
Out[303]: 
<10x10 sparse matrix of type '<class 'numpy.float64'>'
    with 66 stored elements in Compressed Sparse Row format>
In [304]: M.T.dot(M)
Out[304]: 
<10000x10000 sparse matrix of type '<class 'numpy.float64'>'
    with 100310 stored elements in Compressed Sparse Column format>

M的稀疏度为0.01，Out[304]的稀疏度为.001，但非零值仍然多100倍。有了一个（1226880，1226880）矩阵，我可以轻松想象出即使内存稀疏也无法容纳内存的情况。

点积可以执行多大的scipy.csr_matrix矩阵？

1 个答案: