Question

我有5个numpy数组a,b,c,d和e全部定义为：

array([1, 2, 3, 4, 5, 6, 7, 8, 9])

我正在保存这些数组：

np.savez_compressed('tmp/test',a=a,b=b,c=c,d=d,e=e)

这会导致创建一个文件test.npz。

但是我在尝试加载数据时遇到了问题（如例here所示）：

>>> f=np.load('tmp/test.npz')
>>> f.files
['a', 'c', 'b', 'e', 'd']
>>> f['a']

给出一大串错误，结尾于：

File "C:\Python27\lib\compiler\transformer.py", line 754, in atom_lbrace
return self.com_dictorsetmaker(nodelist[1])
File "C:\Python27\lib\compiler\transformer.py", line 1214, in com_dictorsetmaker
assert nodelist[0] == symbol.dictorsetmaker
AssertionError

我考虑过使用pickle。但是，这会导致文件大小是.npz文件的四倍，所以我想使用savez或savez_compressed。

有没有人知道我做错了什么，或建议采用其他方法？

这是一个会产生错误的脚本：

def saver():
    import numpy as np
    a= np.arange(1,10)
    b=a
    c=a
    d=a
    e=a
    np.savez_compressed('tmp/test',a=a,b=b,c=c,d=d,e=e)
    f=np.load('tmp/test.npz')
    print f.files
    print f['a']

以下是完整的追溯：

Traceback (most recent call last):
  File "<pyshell#16>", line 1, in <module>
    saver.saver()
  File "C:\Python27\saver.py", line 14, in saver
    print f['a']
  File "C:\Python27\lib\site-packages\numpy\lib\npyio.py", line 241, in __getitem__
    return format.read_array(value)
  File "C:\Python27\lib\site-packages\numpy\lib\format.py", line 440, in read_array
    shape, fortran_order, dtype = read_array_header_1_0(fp)
  File "C:\Python27\lib\site-packages\numpy\lib\format.py", line 336, in read_array_header_1_0
    d = safe_eval(header)
  File "C:\Python27\lib\site-packages\numpy\lib\utils.py", line 1156, in safe_eval
    ast = compiler.parse(source, mode="eval")
  File "C:\Python27\lib\compiler\transformer.py", line 53, in parse
    return Transformer().parseexpr(buf)
  File "C:\Python27\lib\compiler\transformer.py", line 132, in parseexpr
    return self.transform(parser.expr(text))
  File "C:\Python27\lib\compiler\transformer.py", line 124, in transform
    return self.compile_node(tree)
  File "C:\Python27\lib\compiler\transformer.py", line 159, in compile_node
    return self.eval_input(node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 194, in eval_input
    return Expression(self.com_node(nodelist[0]))
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 578, in testlist
    return self.com_binary(Tuple, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 596, in test
    then = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 610, in or_test
    return self.com_binary(Or, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 615, in and_test
    return self.com_binary(And, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 619, in not_test
    result = self.com_node(nodelist[-1])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 626, in comparison
    node = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 659, in expr
    return self.com_binary(Bitor, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 663, in xor_expr
    return self.com_binary(Bitxor, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 667, in and_expr
    return self.com_binary(Bitand, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 671, in shift_expr
    node = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 683, in arith_expr
    node = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 695, in term
    node = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 715, in factor
    node = self.lookup_node(nodelist[-1])(nodelist[-1][1:])
  File "C:\Python27\lib\compiler\transformer.py", line 727, in power
    node = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 739, in atom
    return self._atom_dispatch[nodelist[0][0]](nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 754, in atom_lbrace
    return self.com_dictorsetmaker(nodelist[1])
  File "C:\Python27\lib\compiler\transformer.py", line 1214, in com_dictorsetmaker
    assert nodelist[0] == symbol.dictorsetmaker
AssertionError

Answer 1

无法在Linux或Mac上重现您的问题（Python 2.7，numpy 1.6.1 / 1.7.1）

但是，我注意到你使用相对路径来保存文件tmp/test.npz。这是故意的吗？在我的回忆中，最新版本的Windows对试图在某些目录中创建的新文件应用程序（如'/ Program Files /'）有一些特殊处理 - 它会将它们移开但仍然告诉应用程序它们在某些情况下存在。这似乎不太可能是这种情况，但是你可以尝试保存文件的绝对路径吗？

顺便说一句：作为ZIP存档（savez savez_compressed创建）的替代方法，您可以尝试使用'LZMAFile'作为文件对象的pickle。它提供了非常好的压缩率（但它可能很慢，并且在压缩/保存文件时需要更多的内存和时间）;

它被用作任何其他文件对象包装器，类似于（加载pickle数据）：

from lzma import LZMAFile
import cPickle as pickle

if fileName.endswith('.xz'):
    dataFile = LZMAFile(fileName,'r')
else:
    dataFile = file(fileName, 'ro')     
data = pickle.load(dataFile)

Answer 2

在MAC OS和Windows上使用numpy 1.7.1 / 1.8.0和python 2.7.6时，我确实遇到了同样的问题（AssertionError）。但是我用python 2.7.5切换到linux后问题自动修复了。然后我在MACOS和Windows上重新安装python 2.7.5，所有问题都消失了。基本上问题是python而不是numpy，因为编译器正在发送警报。所以版本可能很重要。

但是虽然npy是numpy的可序列化类型，但我认为即使对于大型矩阵使用savez_compressed，该文件也不够小。

希望你的问题跟我的一样。

在numpy.savez之后使用np.load进行断言错误

2 个答案: