Question

该文件包含：

1 19 15 36 23 18 39 
2 36 23 4 18 26 9
3 35 6 16 11

由此我想提取列表如下：

L = [1,19,15,36,23,18,19,2,36........... ect.]

最有效的方法是什么？

Answer 1

您可以使用itertools.chain，拆分每一行并映射到整数：

from itertools import chain
with open("in.txt") as f:
    print(list((map(int,chain.from_iterable(line.split() for line in f)))))
[1, 19, 15, 36, 23, 18, 39, 2, 36, 23, 4, 18, 26, 9, 3, 35, 6, 16, 11]

对于python2，请使用itertools.imap而不是map。使用带有map和itertools.chain的链可以避免同时将所有文件读入内存，这是.read将要执行的操作。

文件中python3的某些时间与输入* 1000相同：

In [5]: %%timeit
with open("ints.txt","r") as f:
    list(map(int,re.split(r"\s+",f.read())))
   ...: 
100 loops, best of 3: 8.55 ms per loop

In [6]: %%timeit                                                
with open("ints.txt","r") as f:
    list((map(int, chain.from_iterable(line.split() for line in f))))
   ...: 
100 loops, best of 3: 5.76 ms per loop

In [7]: %%timeit
...: with open("ints.txt","r") as f:
...:      [int(i) for i in f.read().split()]
...: 
100 loops, best of 3: 5.82 ms per loop

因此，itertools匹配list comp但使用的内存要少得多。

对于python2：

In [3]: %%timeit                                                
with open("ints.txt","r") as f:
     [int(i) for i in f.read().split()]
   ...: 
100 loops, best of 3: 7.79 ms per loop

In [4]: %%timeit                                                
with open("ints.txt","r") as f:
    list(imap(int, chain.from_iterable(line.split() for line in f)))
   ...: 
100 loops, best of 3: 8.03 ms per loop

In [5]: %%timeit                                                
with open("ints.txt","r") as f:
    list(imap(int,re.split(r"\s+",f.read())))
   ...: 
100 loops, best of 3: 10.6 ms per loop

列表组件速度稍微快一点，但是再次使用更多内存，如果您要使用读取拆分方法读取全部内存，则imap再次是最快的：

In [6]: %%timeit
   ...: with open("ints.txt","r") as f:
   ...:      list(imap(int, f.read().split()))
   ...: 
100 loops, best of 3: 6.85 ms per loop

python3和map相同：

In [4]: %%timeit                                                
with open("ints.txt","r") as f:
     list(map(int,f.read().split()))
   ...: 
100 loops, best of 3: 4.41 ms per loop

因此，如果你关心速度，请使用list(map(int,f.read().split()))或list(imap(int,f.read().split()))方法如果记忆也是一个问题，请将它与链条结合起来。链接方法的另一个优点是，如果将内存传递给函数或迭代，则可以直接传递链对象，因此您根本不需要将所有数据保存在内存中。

最后一个小优化是在文件对象上映射str.split：

In [5]: %%timeit
with open("ints.txt", "r") as f:
    list((map(int, chain.from_iterable(map(str.split, f)))))
   ...: 
100 loops, best of 3: 5.32 ms per loop

Answer 2

with open('yourfile.txt') as f:
    your_list = f.read().split()

将其强制转换为整数。您可以使用列表compregension：

your_list = [int(i) for i in f.read().split()]

当无法输出值时，这可能会导致异常。

Answer 3

f=open("output.txt","r")
import re
print map(int,re.split(r"\s+",f.read()))
f.close()

您可以使用re.split返回列表，map返回int。

Answer 4

如果您可以使用numpy库，则另一种方法是使用np.fromstring()将文件.read()作为输入，例如 -

import numpy as np
with open('file.txt','r') as f:
    lst = np.fromstring(f.read(),sep=' ',dtype=int)

最后lst将是一个numpy数组，如果你想要一个python列表，请使用list(lst)

numpy.fromstring()总是返回一维数组，当你将空格作为分隔符时，它会忽略额外的空格，包括换行符。

示例/演示 -

In [39]: import numpy as np

In [40]: with open('a.txt','r') as f:
   ....:     lst = np.fromstring(f.read(),sep=' ',dtype=int)
   ....:

In [41]: lst
Out[41]:
array([ 1, 19, 15, 36, 23, 18, 39,  2, 36, 23,  4, 18, 26,  9,  3, 35,  6,
       16, 11])

In [42]: list(lst)
Out[42]: [1, 19, 15, 36, 23, 18, 39, 2, 36, 23, 4, 18, 26, 9, 3, 35, 6, 16, 11]

性能测试 -

In [47]: def func1():
   ....:     with open('a.txt','r') as f:
   ....:         lst = np.fromstring(f.read(),sep=' ',dtype=int)
   ....:         return list(lst)
   ....:
In [37]: def func2():
   ....:     with open('a.txt','r') as f:
   ....:         return list((map(int,chain.from_iterable(line.split() for line in f))))
   ....:

In [54]: def func3():
   ....:     with open('a.txt','r') as f:
   ....:         return np.fromstring(f.read(),sep=' ',dtype=int)
   ....:

In [55]: %timeit func3()
10000 loops, best of 3: 183 µs per loop

In [56]: %timeit func1()
10000 loops, best of 3: 194 µs per loop

In [57]: %timeit func2()
10000 loops, best of 3: 212 µs per loop

如果你对numpy.ndarray没有问题（与列表没有什么不同），那就更快了。

Answer 5

您可以使用re.findall。

import re
with open(file) as f:
    print map(int, re.findall(r'\d+', f.read()))

在Python中从文件创建列表

5 个答案: