使用numpy.load从文件加载压缩数据(.npz)

时间:2013-08-14 11:58:12

标签: python arrays numpy file-io

我有一个数组:

>>> data = np.ones((1,3,128))

我使用savez_compressed将其保存到文件中:

>>> with open('afile','w') as f:
        np.savez_compressed(f,data=data)

当我尝试加载它时,我似乎无法访问数据:

>>> with open('afile','r') as f:
        b=np.load(f)
>>> b.files
['data']
>>> b['data']

Traceback (most recent call last):
  File "<pyshell#196>", line 1, in <module>
    b['data']
  File "C:\Python27\lib\site-packages\numpy\lib\npyio.py", line 238, in __getitem__
    bytes = self.zip.read(key)
  File "C:\Python27\lib\zipfile.py", line 828, in read
    return self.open(name, "r", pwd).read()
  File "C:\Python27\lib\zipfile.py", line 853, in open
    zef_file.seek(zinfo.header_offset, 0)
ValueError: I/O operation on closed file

我做错了什么?

修改

按照@Saullo Castro的回答我尝试了这个:

>>> np.savez_compressed('afile.npz',data=data)
>>> b=np.load('afile.npz')
>>> b.files
['data']
>>> b['data']

并收到以下错误:

Traceback (most recent call last):
  File "<pyshell#253>", line 1, in <module>
    b['data']
  File "C:\Python27\lib\site-packages\numpy\lib\npyio.py", line 241, in __getitem__
    return format.read_array(value)
  File "C:\Python27\lib\site-packages\numpy\lib\format.py", line 440, in read_array
    shape, fortran_order, dtype = read_array_header_1_0(fp)
  File "C:\Python27\lib\site-packages\numpy\lib\format.py", line 336, in read_array_header_1_0
    d = safe_eval(header)
  File "C:\Python27\lib\site-packages\numpy\lib\utils.py", line 1156, in safe_eval
    ast = compiler.parse(source, mode="eval")
  File "C:\Python27\lib\compiler\transformer.py", line 53, in parse
    return Transformer().parseexpr(buf)
  File "C:\Python27\lib\compiler\transformer.py", line 132, in parseexpr
    return self.transform(parser.expr(text))
  File "C:\Python27\lib\compiler\transformer.py", line 124, in transform
    return self.compile_node(tree)
  File "C:\Python27\lib\compiler\transformer.py", line 159, in compile_node
    return self.eval_input(node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 194, in eval_input
    return Expression(self.com_node(nodelist[0]))
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 578, in testlist
    return self.com_binary(Tuple, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 596, in test
    then = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 610, in or_test
    return self.com_binary(Or, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 615, in and_test
    return self.com_binary(And, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 619, in not_test
    result = self.com_node(nodelist[-1])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 626, in comparison
    node = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 659, in expr
    return self.com_binary(Bitor, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 663, in xor_expr
    return self.com_binary(Bitxor, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 667, in and_expr
    return self.com_binary(Bitand, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 671, in shift_expr
    node = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 683, in arith_expr
    node = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 695, in term
    node = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 715, in factor
    node = self.lookup_node(nodelist[-1])(nodelist[-1][1:])
  File "C:\Python27\lib\compiler\transformer.py", line 727, in power
    node = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 739, in atom
    return self._atom_dispatch[nodelist[0][0]](nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 754, in atom_lbrace
    return self.com_dictorsetmaker(nodelist[1])
  File "C:\Python27\lib\compiler\transformer.py", line 1214, in com_dictorsetmaker
    assert nodelist[0] == symbol.dictorsetmaker
AssertionError

编辑2

上述错误发生在IDLE中。它使用Ipython。

4 个答案:

答案 0 :(得分:13)

使用numpy.load can pass the file name时,如果扩展名为.npz,则会首先进行解压缩:

np.savez_compressed('filename.npz', array1=array1, array2=array2)
b = np.load('filename.npz')

并执行b['array1']等等以从每个数组中检索数据......

答案 1 :(得分:4)

  

您还可以使用f属性,这样就可以了   np.ndarray

images = np.load('images.npz')
images = images.f.arr_0

.npz文件中的数组的名称/键(例如arr_0)可以通过

找到
images.keys()

注意:加载的文档字符串中未记录f属性。当load读取npz文件时,它返回class NpzFile的实例。此课程以numpy.lib.npyio.NpzFile的形式提供。 NpzFile类的docstring描述了f属性。 (截至撰写本文时,可以找到该类的源代码here

答案 2 :(得分:1)

尝试将文件打开为二进制文件:

with open('afile','rb') as f:

答案 3 :(得分:1)

在MAC OS和Windows上使用numpy 1.7.1 / 1.8.0和python 2.7.6时,我确实遇到了同样的问题(AssertionError)。但是我用python 2.7.5切换到linux后问题自动修复了。然后我在MACOS和Windows上重新安装python 2.7.5,所有问题都消失了。 基本上问题是python而不是numpy,因为编译器正在发送警报。所以版本可能很重要。

但是虽然npy是numpy的可序列化类型,但我认为即使对于大型矩阵使用savez_compressed,该文件也不够小。

希望你的问题与我的问题相同