Python namedtuple作为apply_async(..)回调的参数

时间:2017-01-24 14:54:28

标签: python python-2.7 python-3.x python-multithreading

我正在写一个简短的程序,我想异步调用一个函数,这样它就不会阻塞调用者。为此,我使用了python' Table3 ID Salary1 Salary2 Salary3 Salary4 Salary5 1 10 11 12 0 0 2 12 13 0 0 0 3 10 0 0 0 0 4 10 12 0 0 0 5 10 0 0 0 0 模块中的Pool

在异步调用的函数中,我想返回multiprocessing以符合我程序其余部分的逻辑,但我发现namedtuple似乎不是从生成的进程传递给回调的受支持类型(可能因为它无法被pickle)。这是问题的最低限度。

namedtuple

有没有人知道是否有办法将异常进程的from multiprocessing import Pool from collections import namedtuple logEntry = namedtuple("LogEntry", ['logLev', 'msg']) def doSomething(x): # Do actual work here logCode = 1 statusStr = "Message Here" return logEntry(logLev=logCode, msg=statusStr) def callbackFunc(result): print(result.logLev) print(result.msg) def userAsyncCall(): pool = Pool() pool.apply_async(doSomething, [1,2], callback=callbackFunc) if __name__ == "__main__": userAsyncCall() # Nothing is printed # If this is uncommented, the logLev and status are printed as expected: # y = logEntry(logLev=2, msg="Hello World") # callbackFunc(y) 返回值传递给回调?对于我正在做的事情,是否有更好/更多的pythonic方法?

2 个答案:

答案 0 :(得分:3)

没有打印的原因是apply_async无声地失败。顺便说一句,我认为这是一种不好的行为,只会让人感到困惑。您可以传递error_callback来处理错误。

def errorCallback(exception):
    print(exception)

def userAsyncCall():
    pool = Pool()
    pool.apply_async(doSomething, [1], callback=callbackFunc,  error_callback=errorCallback)
    # You passed wrong arguments. doSomething() takes 1 positional argument.
    # I replace [1,2] with [1].

if __name__ == "__main__":
    userAsyncCall()
    import time
    time.sleep(3) # You need this, otherwise you will never see the output.

当你来到这里时,输出是

Error sending result: 'LogEntry(logLev=1, msg='Message Here')'. Reason: 'PicklingError("Can't pickle <class '__mp_main__.LogEntry'>: attribute lookup LogEntry on __mp_main__ failed",)'

PicklingError! 你是对的,namedtuple无法从生成的进程传递给回调。

也许这不是一种更可接受的方式,但您可以发送dict而不是namedtuple

DagHøidahl纠正后,可以通过名字小组。以下行有效。

LogEntry = namedtuple("LogEntry", ['logLev', 'msg'])

答案 1 :(得分:3)

问题是webpack-dev-server --progress --hot及其namedtuple()参数的返回值不同。也就是说,命名元组的类定义与您给出的变量名称之间存在不匹配。你需要两个匹配:

typename

并相应更新LogEntry = namedtuple("LogEntry", ['logLev', 'msg']) 中的return语句。

完整代码:

doSomething()

(要查看课程定义,请将from multiprocessing import Pool from collections import namedtuple LogEntry = namedtuple("LogEntry", ['logLev', 'msg']) def doSomething(x): # Do actual work here logCode = 1 statusStr = "Message Here" return LogEntry(logLev=logCode, msg=statusStr) def callbackFunc(result): print(result.logLev) print(result.msg) def userAsyncCall(): pool = Pool() return pool.apply_async(doSomething, [1], callback=callbackFunc) if __name__ == "__main__": c = userAsyncCall() # To see whether there was an exception, you can attempt to get() the AsyncResult object. # print c.get() 添加到verbose=True。)