如何处理Python多个池中的所有错误?

时间:2015-01-26 13:18:29

标签: python python-2.7 multiprocessing

表明问题我准备了简单的代码:

from multiprocessing import Pool


class MyError(Exception):

    def __str__(self):
        return repr("Error msg: " + self.args[0])

def pool_function(msg):
    print msg
    raise MyError(msg)
    return 0

def some_function():
    my_pool = Pool(2)
    msg = ['first', 'second']

    my_pool.map(pool_function, msg)

if __name__ == '__main__':
    try:
        some_function()
    except MyError, msg:
        print msg

在这个特定的例子作为输出我得到:

first
second
'Error msg: first'

但我需要:

first
second
'Error msg: first'
'Error msg: second'

问题是,在主函数的级别上,我尝试访问所有唯一的错误消息,而不是在pool_function的级别上处理它,但我只得到第一个错误消息。不幸的是,我工作的真实代码要复杂得多,所以很难用这个代码的结构做一些事情,这在实例中得到了证明。我需要一些简洁明了的解决方案来获取所有错误消息并在主函数级别上处理它。

感谢任何解决方案的建议。

1 个答案:

答案 0 :(得分:1)

您必须将try~ except放入pool_function而不是__main__。如果不是,__main__将在第一个之后停止,除了提升并且没有机会让第二个运行。这是你正在尝试的:

def pool_function(msg):
    print msg
    try:
        raise MyError(msg)
    except:
        return MyError(msg)


def some_function():
    my_pool = Pool(2)
    msg = ['first', 'second']

    return my_pool.map(pool_function, msg)

if __name__ == '__main__':
    try:
       msg= some_function()
    except MyError, msg:
        print msg

它有效,但似乎不是一个好方法,所以:

def pool_function(msg):
    print msg
    try:
        # do something
        raise MyError(msg)
    except:
        return 0,MyError(msg)
    else:
        return 1,# some result

def some_function():
    my_pool = Pool(2)
    msg = ['first', 'second']
    return return my_pool.map(pool_function, msg)

if __name__ == '__main__':
    msg = some_function()
    for result in msg:
        if result[0]:
            # do something when it run successfully
        elif not result[0]:
            print result[1]
            # do something when it got errors