迭代Python中每1000行文本文件

Question

我想迭代文本文件的每1000行。我曾经做过类似于数据库的事情，在那里我首先为每1000行编写一个新的id，并迭代了它。现在我想用文本文件本身来做。是否有一些pythonic方式来做到这一点？我到目前为止才来。

import pandas as pd

input_file = 'text.csv'
my_input = pd.read_csv(input_file, sep = ';')
length = my_input.shape[0]
start = 0
end = 999
#for the length of the whole document take the lines in range(start,end)
   do stuff
   start =+ 1000
   end =+ 1000

Answer 1

它似乎适用于大火库。

import pandas as pd

input_file = 'text.csv'  
my_input = pd.read_csv(input_file, sep = ';', names=['a', 'b', 'c']
for chunk in blaze.odo(my_input, target=bz.chunks(pd.DataFrame), chunksize=1000):
    for index, row in chunk.iterrows():
            variable1 = row['a']
            variable1 = row['b']
            do stuff