在Python中循环使用同一个生成器安全环路中的生成器吗?

时间:2016-05-17 18:47:39

标签: python generator

据我所知,Python中的for x in a_generator: foo(x)循环大致相当于:

try:
    while True:
        foo(next(a_generator))
except StopIteration:
    pass

这表明这样的事情:

for outer_item in a_generator:
    if should_inner_loop(outer_item):
        for inner_item in a_generator:
            foo(inner_item)
            if stop_inner_loop(inner_item): break
    else:
        bar(outer_item)

会做两件事:

  1. 不提出任何例外,段错误或类似的内容
  2. 迭代y,直到x should_inner_loop(x)返回truthy,然后在内部for中循环,直到stop_inner_loop(thing)返回true。然后,外部循环恢复,其中内部停止
  3. 从我公认的不太好的测试中,它似乎表现如上。但是,我无法在规范中找到任何保证此行为在解释器中保持不变的内容。有没有说过或暗示我可以确定它总是这样的?它会导致错误,还是以其他方式执行? (即做一些除上述内容之外的事情

    N.B。上面的代码取自我自​​己的经验;我不知道它是否真的准确。这就是我要问的原因。

3 个答案:

答案 0 :(得分:6)

TL; DR:CPython是安全的(但我找不到任何相关规范),虽然它可能不会做你想做的事。

首先,让我们谈谈你的第一个假设,即等价。

for循环实际调用对象上的第一个iter(),然后对其结果运行next(),直到获得StopIteration

这是相关的字节码(Python的低级形式,由解释器本身使用):

>>> import dis
>>> def f():
...  for x in y:
...   print(x)
... 
>>> dis.dis(f)
  2           0 SETUP_LOOP              24 (to 27)
              3 LOAD_GLOBAL              0 (y)
              6 GET_ITER
        >>    7 FOR_ITER                16 (to 26)
             10 STORE_FAST               0 (x)

  3          13 LOAD_GLOBAL              1 (print)
             16 LOAD_FAST                0 (x)
             19 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
             22 POP_TOP
             23 JUMP_ABSOLUTE            7
        >>   26 POP_BLOCK
        >>   27 LOAD_CONST               0 (None)
             30 RETURN_VALUE

GET_ITER调用iter(y)(它本身调用y.__iter__())并将其结果推送到堆栈(将其视为一堆本地未命名变量),然后进入{{ {1}},调用FOR_ITER(它本身调用next(<iterator>)),然后执行循环内的代码,<iterator>.__next__()使执行返回JUMP_ABSOLUTE

现在,为安全起见:

以下是生成器的方法:https://hg.python.org/cpython/file/101404/Objects/genobject.c#l589 正如您在line 617所看到的那样,FOR_ITER的实施是__iter__(),您可以找到here的实现。 PyObject_SelfIter只返回对象(即生成器)本身。

因此,当您嵌套两个循环时,两者都迭代在同一个迭代器上。 而且,正如你所说,他们只是在上面打PyObject_SelfIter,所以它是安全的。

但要小心:内部循环将消耗外部循环不会消耗的项目。 即使这是你想做的事情,它也可能不太可读。

如果这不是您想要做的,请考虑itertools.tee(),它缓冲迭代器的输出,允许您迭代其输出两次(或更多)。只有当tee迭代器在输出流中保持彼此靠近时,这才有效。如果一个tee迭代器在使用另一个之前将完全耗尽,那么最好只需在迭代器上调用next()来实现其中的列表。

答案 1 :(得分:3)

不,它不安全(因为我们没有获得我们可能预期的结果)。

考虑一下:

{
    "metadata": {
        "version": 4.3,
        "type": "Object",
        "generator": "ObjectExporter"
    },
    "geometries": [
        {
            "uuid": "9848FD0D-E02D-4D2E-BB80-93CD4CA27AFF",
            "type": "Geometry",
            "data": {
                "vertices": [-100,0,-100,100,0,-100,-100,0,-100,-100,0,100,-100,0,-90,100,0,-90,-90,0,-100,-90,0,100,-100,0,-80,100,0,-80,-80,0,-100,-80,0,100,-100,0,-70,100,0,-70,-70,0,-100,-70,0,100,-100,0,-60,100,0,-60,-60,0,-100,-60,0,100,-100,0,-50,100,0,-50,-50,0,-100,-50,0,100,-100,0,-40,100,0,-40,-40,0,-100,-40,0,100,-100,0,-30,100,0,-30,-30,0,-100,-30,0,100,-100,0,-20,100,0,-20,-20,0,-100,-20,0,100,-100,0,-10,100,0,-10,-10,0,-100,-10,0,100,-100,0,0,100,0,0,0,0,-100,0,0,100,-100,0,10,100,0,10,10,0,-100,10,0,100,-100,0,20,100,0,20,20,0,-100,20,0,100,-100,0,30,100,0,30,30,0,-100,30,0,100,-100,0,40,100,0,40,40,0,-100,40,0,100,-100,0,50,100,0,50,50,0,-100,50,0,100,-100,0,60,100,0,60,60,0,-100,60,0,100,-100,0,70,100,0,70,70,0,-100,70,0,100,-100,0,80,100,0,80,80,0,-100,80,0,100,-100,0,90,100,0,90,90,0,-100,90,0,100,-100,0,100,100,0,100,100,0,-100,100,0,100],
                "normals": [],
                "faces": []
            }
        },
        {
            "uuid": "DCC723E0-C1E0-4CB4-B047-56AF8FCDD3E2",
            "type": "BoxGeometry",
            "width": 4,
            "height": 4,
            "depth": 4
        },
        {
            "uuid": "AE0EA8E4-47AC-4D29-AD68-CE00181F9E21",
            "type": "BoxGeometry",
            "width": 4,
            "height": 4,
            "depth": 4
        },
        {
            "uuid": "0AE6C8FD-83BD-4946-BFCB-F767B01FE579",
            "type": "BoxGeometry",
            "width": 4,
            "height": 4,
            "depth": 4
        },
        {
            "uuid": "6B40660B-CF8B-4CB8-AD55-E0F959C5979C",
            "type": "BoxGeometry",
            "width": 4,
            "height": 4,
            "depth": 4
        }],
    "materials": [
        {
            "uuid": "F174D367-B14B-47C9-9818-1D1A62540882",
            "type": "LineBasicMaterial"
        },
        {
            "uuid": "83848195-1452-40DA-827F-A498C1D870F9",
            "type": "MeshPhongMaterial",
            "color": 16777215,
            "ambient": 16777215,
            "emissive": 0,
            "specular": 1118481,
            "shininess": 30,
            "opacity": 0.8,
            "transparent": true
        },
        {
            "uuid": "37C5C4FD-3402-4FE4-877D-E66BD18D797E",
            "type": "MeshPhongMaterial",
            "color": 16777215,
            "ambient": 16777215,
            "emissive": 0,
            "specular": 1118481,
            "shininess": 30,
            "opacity": 0.8,
            "transparent": true
        },
        {
            "uuid": "FC6E4C1C-8191-4CE9-B687-A29C13EA9400",
            "type": "MeshPhongMaterial",
            "color": 16777215,
            "ambient": 16777215,
            "emissive": 0,
            "specular": 1118481,
            "shininess": 30,
            "opacity": 0.8,
            "transparent": true
        },
        {
            "uuid": "CE06172B-3277-4B91-B113-3BC45963EE2E",
            "type": "MeshPhongMaterial",
            "color": 16777215,
            "ambient": 16777215,
            "emissive": 0,
            "specular": 1118481,
            "shininess": 30,
            "opacity": 0.8,
            "transparent": true
        }],
    "object": {
        "uuid": "FDF89002-FA43-4B30-BEB1-8548BFF8592D",
        "type": "Scene",
        "matrix": [1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1],
        "children": [
            {
                "uuid": "5E79AA2B-C280-4A2B-9312-440F701CBEE3",
                "type": "PerspectiveCamera",
                "fov": 50,
                "aspect": 1,
                "near": 0.1,
                "far": 2000,
                "matrix": [0.8944271802902222,-1.3716013880227251e-9,-0.4472135901451111,0,-0.15064871311187744,0.9415544867515564,-0.3012974262237549,0,0.4210759699344635,0.3368607759475708,0.842151939868927,0,50,50,100,1]
            },
            {
                "uuid": "3984192A-DD7F-4689-A076-6770131B23AD",
                "type": "Line",
                "name": "Grid",
                "geometry": "9848FD0D-E02D-4D2E-BB80-93CD4CA27AFF",
                "material": "F174D367-B14B-47C9-9818-1D1A62540882",
                "matrix": [1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1]
            },
            {
                "uuid": "ACB0E066-532C-4444-9FA7-181C76BAAB95",
                "type": "AmbientLight",
                "color": 5592405,
                "matrix": [1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1]
            },
            {
                "uuid": "FFDE61E5-CA82-4BC9-B406-1B7A9006D1DD",
                "type": "DirectionalLight",
                "color": 16777215,
                "intensity": 0.8,
                "matrix": [1,0,0,0,0,1,0,0,0,0,1,0,0.5773502588272095,0.5773502588272095,0.5773502588272095,1]
            },
            {
                "uuid": "A94B706F-7329-48A5-9CF8-040795E57314",
                "type": "SpotLight",
                "color": 16777215,
                "intensity": 0.6,
                "distance": 0,
                "angle": 1.0471975511965976,
                "exponent": 10,
                "matrix": [1,0,0,0,0,1,0,0,0,0,1,0,-30,30,-100,1]
            },
            {
                "uuid": "6608DD55-558B-4718-B953-F686D42B3744",
                "type": "Mesh",
                "geometry": "DCC723E0-C1E0-4CB4-B047-56AF8FCDD3E2",
                "material": "83848195-1452-40DA-827F-A498C1D870F9",
                "matrix": [1,0,0,0,0,1,0,0,0,0,1,0,-9.931706428527832,0,-9.857349395751953,1]
            },
            {
                "uuid": "25A41D96-791D-4FBE-9CB9-5AEC0ECBE991",
                "type": "Mesh",
                "geometry": "AE0EA8E4-47AC-4D29-AD68-CE00181F9E21",
                "material": "37C5C4FD-3402-4FE4-877D-E66BD18D797E",
                "matrix": [1,0,0,0,0,1,0,0,0,0,1,0,2.9669816493988037,0,5.327826976776123,1]
            },
            {
                "uuid": "B6FE254F-BCA8-41B1-A63C-4CB7CAA6F734",
                "type": "Mesh",
                "geometry": "0AE6C8FD-83BD-4946-BFCB-F767B01FE579",
                "material": "FC6E4C1C-8191-4CE9-B687-A29C13EA9400",
                "matrix": [1,0,0,0,0,1,0,0,0,0,1,0,1.5996896028518677,0,-7.717252254486084,1]
            },
            {
                "uuid": "0EFA29CB-D365-4942-B54B-D43FFFB22D3E",
                "type": "Mesh",
                "geometry": "6B40660B-CF8B-4CB8-AD55-E0F959C5979C",
                "material": "CE06172B-3277-4B91-B113-3BC45963EE2E",
                "matrix": [1,0,0,0,0,1,0,0,0,0,1,0,8.166288375854492,0,14.097293853759766,1]
            }]
    }
}

当然,我们将打印0到19张。

现在让我们添加一些代码:

a = (_ for _ in range(20))
for num in a:
    print(num)

唯一要打印的是a = (_ for _ in range(20)) for num in a: for another_num in a: pass print(num) 。 到达外循环的第二次迭代时,内循环已经耗尽了生成器。

我们也可以这样做:

0

如果它是安全的,我们预计会打印20到0到19,但我们实际上只打印了一次,原因与上面提到的相同。

答案 2 :(得分:2)

这不是你问题的答案,但我建议不要这样做,因为代码不可读。我花了一段时间才发现您使用y两次,即使这是您问题的全部内容。不要让未来的读者对此感到困惑。当我看到一个嵌套的循环时,我并不期待你做了什么,我的大脑也很难看到它。

我会这样做:

def generator_with_state(y):
    state = 0
    for x in y:
        if isinstance(x, special_thing):
            state = 1
            continue
        elif state == 1 and isinstance(x, signal):
            state = 0
        yield x, state

for x, state in generator_with_state(y):
    if state == 1:
        foo(x)
    else:
        bar(x)