Question

我正在努力处理大型单列数据的数据操作，如testo.dat所示（google Drive认为此文件是视频，但1.6Mb数据文件 - 不用担心）。我到目前为止所做的代码部分工作，您可以在下面查看。

我正在尝试使用我的输入：

加载文本并存储到1d数组 - 工作正常
根据1d数组的长度重新形成一个矩阵 - 也可以正常工作
将单个数组中的值填入新创建的矩阵中，以便将数据排列在网格/矩阵上，并使用行和列所需的数字 - 这是我的实际问题！
最后转置矩阵 - 如果我有一个填充矩阵，这可以正常工作

所以我的问题是：带有矩阵填充的部分（第3点）是否有意义，因为它坚持使用1d数组的第一行？我在其他帖子方法中看到过“切片”和“拆分” - 这些适用于此处吗？

非常感谢任何帮助

代码

import numpy as np
import matplotlib.pyplot as plt

# 1. Load a file with a single column and store into an array 
data = np.loadtxt('testo.dat',skiprows=3) 
bz_array = data[:]                        


# 2. Prepare the matrix by setting up rows and cols which matchs the    length of data
rows = 20
cols = 3600
n=len(data)                              
if (n == (rows*cols)):
    print "rows*cols == length of data" 

# 3. Create the matrix 
shape = (rows,cols) # 72000 lines/20 = 3600 columns
bz_matrix = np.arange(n).reshape(shape)
if np.shape(bz_matrix) == shape:
    print "shape is ok"

# 4. How to fill this matrix with minimum of memory and cpu ? 
line=0

while line < n:
    for i in range(cols):
        for j in range(rows):
            bz_matrix=bz_array[line]
            print bz_matrix
    line+=line


# 5. Transpose the matrix 
bz_matrix_transposed = bz_matrix.transpose()
if (np.shape(bz_matrix_transposed) == (cols,rows)):
    print "transposed shape is ok" 

# 6. I would like to have also a printed file of the transposed matrix
f = open('out.dat', 'w') 
np.savetxt(f, bz_matrix_transposed, fmt='%10.5f') 


# 7. plot a 2D graph of the transposed matrix,
plt.imshow(bz_matrix_transposed((cols,rows)));
plt.colorbar()
plt.show()

Answer 1

我想更新上面的代码。感谢Georgy（Jan-19,2018），我可以看到实际问题在哪里。

# 1. Load a file with a single column and store into an array 
data = np.loadtxt('testo.dat',skiprows=3) 
bz_array = data[:]                        


# 2. Prepare the matrix by setting up rows and cols which matchs the                 length of data
rows = 20
cols = 3600
n=len(data)                              
if (n == (rows*cols)):
    print "rows*cols == length of data" 

# 3. Create the matrix 
shape = (rows,cols) # 72000 lines/20 = 3600 columns
bz_array.shape=(shape)


# 5. Transpose the matrix 
bz_matrix_transposed = bz_array.transpose()
if (np.shape(bz_matrix_transposed) == (cols,rows)):
    print "transposed shape is ok" 


# 6. I would like to have the finnished data as a seperate file 
bz_finnished = np.asarray(bz_matrix_transposed).ravel()
f = open('out_finnished', 'w') 
np.savetxt(f, bz_finnished, fmt='%10.5f')

如何正确地重塑，填充和转置矩阵并使内存友好

代码

1 个答案: