Question

我希望迭代来自ipython并行映射的一些异步结果。我能找到的唯一方法是迭代结果对象。但是，如果其中一个任务引发异常，则迭代终止。有没有办法做到这一点？请参阅下面的代码，迭代在第二个作业引发异常时终止。

from IPython import parallel

def throw_even(i):
    if i % 2 == 0:
        raise RuntimeError('ERROR: %d' % i)
    return i

rc = parallel.Client()
lview = rc.load_balanced_view() # default load-balanced view

# map onto the engines.
args = range(1, 5)
print args
async_results = lview.map_async(throw_even, range(1, 5), ordered=True)

# get results
args_iter = iter(args)
results_iter = iter(async_results)
while True:
    try:
        arg = args_iter.next()
        result = results_iter.next()
        print 'Job %s completed: %d' % (arg, result)            
    except StopIteration:
        print 'Finished iteration'
        break
    except Exception as e:
        print '%s: Job %d: %s' % (type(e), arg, e)

提供以下输出，在报告作业3和4之前停止

[1, 2, 3, 4]
Job 1 completed: 1
<class 'IPython.parallel.error.RemoteError'>: Job 2: RuntimeError(ERROR: 2)
Finished iteration

有没有办法做到这一点？

Answer 1

此question可能相关。我不明白你为什么要从远程引擎中抛出异常。虽然，如果你确实想这样做，我认为你可以用我回答上述问题的方式来做。我现在看到你已经在评论中意识到了这一点，但无论如何都应该这样做。

def throw_even(i):
    if i%2:
       return i
    raise(RuntimeError('Error %d'%i)

params = range(1,5)

n_cores = len(c.ids)
for n,p in enumerate( params ):
    core = c.ids[n%n_cores]
    calls.append( c[core].apply_async( throw_even, p ) )

#then you get the results

while calls != []:
    for c in calls:
        try:
             result = c.get(1e-3)
             print(result[0])
             calls.remove( c )
             #in the case your call failed, you can apply_async again.
             # and append the call to calls.
        except parallel.TimeoutError:
             pass
        except Exception as e:
             knock_yourself_out(e)

Answer 2

这方面的一个偷偷摸摸的方法是进入AsyncMapResult的内部并抓住_result这是一个结果列表。这对您没有直接帮助，但仅限于以下事实：

tt = async_results._results
fail_indx = [j for j, r in enumerate(tt) if isinstance(r, IPython.parallel.error.RemoteError)]
good_indx = [j for j, r in enumerate(tt) if not isinstance(r, IPython.parallel.error.RemoteError)]

just_the_results =  [r for  r in tt if not isinstance(r, IPython.parallel.error.RemoteError)]

在等待下一个ipython并行映射结果时处理异常

2 个答案: