无法在Python中读取大型CSV

时间:2014-06-04 22:02:55

标签: python csv

我正在使用python 2.7并尝试读取CSV文件的条目。我创建了原始CSV的单独版本,它只有前10行数据,并且使用以下代码,它按照我希望的方式工作,我可以在genfromtxt的“usecols”字段中编辑Z的索引读取我的CSV中的特定列范围。

import numpy as np
import array

Z = array.array('i', (i for i in range(0, 40)))

with open('data/training_edit.csv','r') as f:
    data = np.genfromtxt(f, dtype=float, delimiter=',', names=True, usecols=(Z[0:32])) 
print(data)

但是当我使用原始CSV(250,000行x 33列)的代码时,我得到了这种输出,我不知道为什么:

Traceback (most recent call last):
File "/home/user/PycharmProjects/H-B2/Read.py", line 74, in <module>
data = np.genfromtxt(f, dtype=float, delimiter=',', names=True,usecols=(Z[0:32]))
File "/usr/lib/python2.7/dist-packages/numpy/lib/npyio.py", line 1667, in genfromtxt  
raise ValueError(errmsg)
ValueError: Some errors were detected !

.
.
.   
Line #249991 (got 1 columns instead of 32)
Line #249992 (got 1 columns instead of 32)
Line #249993 (got 1 columns instead of 32)
Line #249994 (got 1 columns instead of 32)
Line #249995 (got 1 columns instead of 32)
Line #249996 (got 1 columns instead of 32)
Line #249997 (got 1 columns instead of 32)
Line #249998 (got 1 columns instead of 32)
Line #249999 (got 1 columns instead of 32)
Line #250000 (got 1 columns instead of 32)

Process finished with exit code 1

(我添加了这些点只是为了缩短实际输出,但你希望得到这一点)

1 个答案:

答案 0 :(得分:0)

是的,我相信您只需要将range(0,32)添加到您的usecols中,如下所示:

  

data = np.genfromtxt(f,dtype = float,delimiter =&#39;,&#39;,   名称=真,usecols =范围(0,32))

我只是为自己想出来。