在itertools中chain和chain.from_iterable有什么区别？

Question

我在互联网上找不到任何有效的例子，我可以看到它们之间的区别以及为什么选择一个而不是另一个。

Answer 1

第一个接受0个或多个参数，每个参数都是一个可迭代的，第二个参数接受一个参数，该参数应该产生迭代：

itertools.chain(list1, list2, list3)

iterables = [list1, list2, list3]
itertools.chain.from_iterable(iterables)

但iterables可以是产生迭代的任何迭代器。

def generate_iterables():
    for i in range(10):
        yield range(i)

itertools.chain.from_iterable(generate_iterables())

使用第二种形式通常是一种方便的情况，但是因为它懒惰地循环输入迭代，它也是链接无限数量的有限迭代器的唯一方法：

def generate_iterables():
    while True:
        for i in range(5, 10):
            yield range(i)

itertools.chain.from_iterable(generate_iterables())

上面的示例将为您提供一个迭代，它产生一个永远不会停止的循环数字模式，但永远不会消耗比单个range()调用所需的内存更多的内存。

Answer 2

他们做的事非常相似。对于少量迭代，itertools.chain(*iterables)和itertools.chain.from_iterable(iterables)执行类似。

from_iterables的关键优势在于能够处理大量（可能无限）的迭代次数，因为在调用时它们都不需要可用。

Answer 3

扩展@martijn-pieters answer

尽管在迭代过程中对内部项目的访问仍然相同，并且在实现方面很明智，

• itertools_chain_from_iterable（即Python中的chain.from_iterable）和
• chain_new（即Python中的chain）

在CPython实现中，都是chain_new_internal的鸭子类型

---

使用chain.from_iterable(x)有什么优化好处，其中x是可迭代的；而主要目的是最终消耗平整的项目清单？

我们可以尝试对它进行基准测试

import random
from itertools import chain
from functools import wraps
from time import time

from tqdm import tqdm

def timing(f):
    @wraps(f)
    def wrap(*args, **kw):
        ts = time()
        result = f(*args, **kw)
        te = time()
        print('func:%r args:[%r, %r] took: %2.4f sec' % (f.__name__, args, kw, te-ts))
        return result
    return wrap

def generate_nm(m, n):
    # Creates m generators of m integers between range 0 to n.
    yield iter(random.sample(range(n), n) for _ in range(m))
    

def chain_star(x):
    # Stores an iterable that will unpack and flatten the list of list.
    chain_x = chain(*x)
    # Consumes the items in the flatten iterable.
    for i in chain_x:
        pass

def chain_from_iterable(x):
    # Stores an iterable that will unpack and flatten the list of list.
    chain_x = chain.from_iterable(x)
    # Consumes the items in the flatten iterable.
    for i in chain_x:
        pass


@timing
def versus(f, n, m):
  f(generate_nm(n, m))

P / S：正在运行基准测试...等待结果。

---

结果

chain_star，m = 1000，n = 1000

for _ in range(10):
    versus(chain_star, 1000, 1000)

[输出]：

func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 1000, 1000), {}] took: 0.6494 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 1000, 1000), {}] took: 0.6603 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 1000, 1000), {}] took: 0.6367 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 1000, 1000), {}] took: 0.6350 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 1000, 1000), {}] took: 0.6296 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 1000, 1000), {}] took: 0.6399 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 1000, 1000), {}] took: 0.6341 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 1000, 1000), {}] took: 0.6381 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 1000, 1000), {}] took: 0.6343 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 1000, 1000), {}] took: 0.6309 sec

chain_from_iterable，m = 1000，n = 1000

for _ in range(10):
    versus(chain_from_iterable, 1000, 1000)

[输出]：

func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 1000, 1000), {}] took: 0.6416 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 1000, 1000), {}] took: 0.6315 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 1000, 1000), {}] took: 0.6535 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 1000, 1000), {}] took: 0.6334 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 1000, 1000), {}] took: 0.6327 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 1000, 1000), {}] took: 0.6471 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 1000, 1000), {}] took: 0.6426 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 1000, 1000), {}] took: 0.6287 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 1000, 1000), {}] took: 0.6353 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 1000, 1000), {}] took: 0.6297 sec

chain_star，m = 10000，n = 1000

func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 10000, 1000), {}] took: 6.2659 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 10000, 1000), {}] took: 6.2966 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 10000, 1000), {}] took: 6.2953 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 10000, 1000), {}] took: 6.3141 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 10000, 1000), {}] took: 6.2802 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 10000, 1000), {}] took: 6.2799 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 10000, 1000), {}] took: 6.2848 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 10000, 1000), {}] took: 6.3299 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 10000, 1000), {}] took: 6.2730 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 10000, 1000), {}] took: 6.3052 sec

chain_from_iterable，m = 10000，n = 1000

func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 10000, 1000), {}] took: 6.3129 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 10000, 1000), {}] took: 6.3064 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 10000, 1000), {}] took: 6.3071 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 10000, 1000), {}] took: 6.2660 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 10000, 1000), {}] took: 6.2837 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 10000, 1000), {}] took: 6.2877 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 10000, 1000), {}] took: 6.2756 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 10000, 1000), {}] took: 6.2939 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 10000, 1000), {}] took: 6.2715 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 10000, 1000), {}] took: 6.2877 sec

chain_star，m = 100000，n = 1000

func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 100000, 1000), {}] took: 62.7874 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 100000, 1000), {}] took: 63.3744 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 100000, 1000), {}] took: 62.5584 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 100000, 1000), {}] took: 63.3745 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 100000, 1000), {}] took: 62.7982 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 100000, 1000), {}] took: 63.4054 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 100000, 1000), {}] took: 62.6769 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 100000, 1000), {}] took: 62.6476 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 100000, 1000), {}] took: 63.7397 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 100000, 1000), {}] took: 62.8980 sec

chain_from_iterable，m = 100000，n = 1000

for _ in range(10):
    versus(chain_from_iterable, 100000, 1000)

[输出]：

func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 100000, 1000), {}] took: 62.7227 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 100000, 1000), {}] took: 62.7717 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 100000, 1000), {}] took: 62.7159 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 100000, 1000), {}] took: 62.7569 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 100000, 1000), {}] took: 62.7906 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 100000, 1000), {}] took: 62.6211 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 100000, 1000), {}] took: 62.7294 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 100000, 1000), {}] took: 62.8260 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 100000, 1000), {}] took: 62.8356 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 100000, 1000), {}] took: 62.9738 sec

chain_star，m = 500000，n = 1000

for _ in range(3):
    versus(chain_from_iterable, 500000, 1000)

[输出]：

func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 500000, 1000), {}] took: 314.5671 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 500000, 1000), {}] took: 313.9270 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 500000, 1000), {}] took: 313.8992 sec

chain_from_iterable，m = 500000，n = 1000

for _ in range(3):
    versus(chain_from_iterable, 500000, 1000)

[输出]：

func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 500000, 1000), {}] took: 313.8301 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 500000, 1000), {}] took: 313.8104 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 500000, 1000), {}] took: 313.9440 sec

Answer 4

  

我找不到任何有效的示例...在这里我可以看到它们之间的区别[chain和chain.from_iterable]以及为什么要选择一个而不是另一个

公认的答案是彻底的。对于那些寻求快速申请的人，请考虑展平几个列表：

list(itertools.chain(["a", "b", "c"], ["d", "e"], ["f"]))
# ['a', 'b', 'c', 'd', 'e', 'f']

您可能希望稍后再使用这些列表，因此您可以对列表进行迭代：

iterable = (["a", "b", "c"], ["d", "e"], ["f"])

尝试

但是，将可迭代数传递给chain会产生不平坦的结果：

list(itertools.chain(iterable))
# [['a', 'b', 'c'], ['d', 'e'], ['f']]

为什么？您传入了一个项（元组）。 chain分别需要每个列表。

---

解决方案

在可能的情况下，您可以打开可迭代项的包装：

list(itertools.chain(*iterable))
# ['a', 'b', 'c', 'd', 'e', 'f']

list(itertools.chain(*iter(iterable)))
# ['a', 'b', 'c', 'd', 'e', 'f']

更一般地，使用.from_iterable（因为它也适用于无限迭代器）：

list(itertools.chain.from_iterable(iterable))
# ['a', 'b', 'c', 'd', 'e', 'f']

g = itertools.chain.from_iterable(itertools.cycle(iterable))
next(g)
# "a"

Answer 5

另一种查看方式：

chain(iterable1, iterable2, iterable3, ...)用于当您已经知道自己具有哪些可迭代项时，因此您可以将它们编写为这些逗号分隔的参数。

chain.from_iterable(iterable)用于从另一个迭代器中获取您的迭代器（例如iterable1，iterable2，iterable3）。

Answer 6

查看它的另一种方法是使用chain.from_iterable

当您拥有可迭代的可迭代对象（例如嵌套可迭代对象（或复合可迭代对象））并使用链来实现简单可迭代对象时