如何从csv中随机抽样一百万次

时间:2015-10-04 08:00:07

标签: python random iteration sampling

我有一个很大的csv看起来像这样

claim score
yes   1
yes   1
no    1
no    1
yes   1
...   1
...   1
...   1

得分是完全相同的数字,我需要运行大量随机抽样的大小说(1000)。然后计算是计数的平均百分比

代码如下所示:

#imports
import random
import numpy

TotalYes = 0
csvFile = numpy.genfromtxt("/nas/home/twu/wind/output_1.csv",delimiter=",",dtype=None)
for j in range(1,10001):
    #csv format : claim (Yes/No), value
    #read in your csv file and store in array
    #initialize random number generator
    random.seed()

#create RandomSample array
RandSamples=[]
samplesize = 1000
#Fill RandomSample array with 10000 random samples from cvs array
for i in range(1,1001):
    #for row in csvFile:
    #get a random index within csvFile[]. random num range is 0 to csv array length
    randIndex=random.randint(0,len(csvFile))
    print randIndex
    RandSamples.append(csvFile[randIndex:randIndex+1,:])
#RandSamples1=numpy.asarray(RandSamples)
#get number of 'yes' from RandomSample array
RandYesSample=[]
for i in range(0,1001):

    # check to see if current record is Yes claim or no
    if RandSamples[i:i+1,:1] == "yes":
        #yes, copy value to yes array
        RandYesSample.append (RandSamples[i:i+1,:1])

#get percent of yes in RandomSample array
PercYes = float(len(RandYesSample)) / 1000
TotalYes = TotalYes + PercYes

TotalYes = float(TotalYes) / 10000

print TotalYes  

我遇到的错误是:

if RandSamples[i:i+1,:1] == "yes":...TypeError: list indices must be
integers, not tuple

我无法让它发挥作用。有人可以帮忙吗?

1 个答案:

答案 0 :(得分:0)

切片列表有问题,它应该像[start:end:step]但是你要删除一个逗号:

csvFile[randIndex:randIndex+1,:]

应该是

csvFile[randIndex]

同样在:

if RandSamples[i] == "yes":

RandYesSample.append (RandSamples[i])