Question

我有一个大文件（大约80,000行），我想将每个10行块存储到一个单独的列表中。对于我的前三个10行块：

N=10 #Number of lines per block

with open("file", "r") as myfile:
        profile1 = list(islice(myfile, 0,N))
        profile2 = list(islice(myfile, 0,N))
        profile3 = list(islice(myfile, 0,N))

我希望有这些10行的数百块，所以这显然不是一个很好的方法。

如何将列表生成和islice函数合并到循环中？

提前谢谢！

Answer 1

使用以下内容：

with open('file', 'r') as f:
    lines = f.readlines()
chunks = [lines[item:item+10] for item in range(0, len(lines), 10)]  # with Python 2 you can use xrange instead of range for large lists

要将每个块转换为数组，请尝试以下操作：

import numpy as np

my_arrays = [np.asarray(chunk) for chunk in chunks]

Answer 2

你可以试试这个：

import numpy as np

# read the file in lines
with open('file.txt','r') as f:
    lines = f.read().splitlines()

# use a list comprehension to split your list in chunks of 10
list_of_lists = [lines[i:i + 10] for i in xrange(0, len(lines), 10)]

# 1st chunks of 10 
print list_of_lists[0]
# 4th chunks of 10
print list_of_lists[3]

# update - turn into arrays
list_of_arrays = []
for i in list_of_lists:
    arr = np.asarray(i)
    list_of_arrays.append(arr)

循环N行文件并存储在multipel列表中

2 个答案: