我已经分发了dask,这是对工作流程(完美)的更大努力的一部分,并且在我的dask工作者中看到了一个错误。工人收到了工作,但很快提出了例外情况:
TypeError: can not serialize 'Variable' object
完整跟踪为:
distributed.protocol.core - CRITICAL - Failed to Serialize
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/distributed/protocol/core.py", line 25, in dumps
small_header, small_payload = dumps_msgpack(msg)
File "/opt/conda/lib/python3.7/site-packages/distributed/protocol/core.py", line 163, in dumps_msgpack
payload = msgpack.dumps(msg, use_bin_type=True)
File "/opt/conda/lib/python3.7/site-packages/msgpack/__init__.py", line 35, in packb
return Packer(**kwargs).pack(o)
File "msgpack/_packer.pyx", line 286, in msgpack._cmsgpack.Packer.pack
File "msgpack/_packer.pyx", line 292, in msgpack._cmsgpack.Packer.pack
File "msgpack/_packer.pyx", line 289, in msgpack._cmsgpack.Packer.pack
File "msgpack/_packer.pyx", line 225, in msgpack._cmsgpack.Packer._pack
File "msgpack/_packer.pyx", line 283, in msgpack._cmsgpack.Packer._pack
TypeError: can not serialize 'Variable' object
distributed.comm.utils - INFO - Unserializable Message: {'op': 'variable_get', 'timeout': 0, 'name': <distributed.variable.Variable object at 0x7fec9eafaef0>, 'client': 'Client-worker-365168fe-ccf9-11ea-800b-0242ac110004', 'reply': True}
distributed.comm.utils - ERROR - can not serialize 'Variable' object
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/distributed/comm/utils.py", line 35, in _to_frames
msg, serializers=serializers, on_error=on_error, context=context
File "/opt/conda/lib/python3.7/site-packages/distributed/protocol/core.py", line 25, in dumps
small_header, small_payload = dumps_msgpack(msg)
File "/opt/conda/lib/python3.7/site-packages/distributed/protocol/core.py", line 163, in dumps_msgpack
payload = msgpack.dumps(msg, use_bin_type=True)
File "/opt/conda/lib/python3.7/site-packages/msgpack/__init__.py", line 35, in packb
return Packer(**kwargs).pack(o)
File "msgpack/_packer.pyx", line 286, in msgpack._cmsgpack.Packer.pack
File "msgpack/_packer.pyx", line 292, in msgpack._cmsgpack.Packer.pack
File "msgpack/_packer.pyx", line 289, in msgpack._cmsgpack.Packer.pack
File "msgpack/_packer.pyx", line 225, in msgpack._cmsgpack.Packer._pack
File "msgpack/_packer.pyx", line 283, in msgpack._cmsgpack.Packer._pack
TypeError: can not serialize 'Variable' object
我已经将其归结为一个非常小的测试用例,我认为这是相当有代表性的。它引起了同样的问题:
from distributed.protocol.core import dumps_msgpack
from distributed.variable import Variable
from distributed.client import Client
client = Client(address='10.72.112.29:38786')
var = Variable(name='test', client=client)
msg = {'op': 'variable_get', 'timeout': 0, 'name': var, 'client': client.id, 'reply': True}
d = dumps_msgpack(msg)
此示例会产生错误消息,但也会显示一些版本不匹配(我不认为是问题所在):
/Users/gdesmarais/git/rightsize/rightsize-venv/lib/python3.7/site-packages/distributed/client.py:1136: VersionMismatchWarning: Mismatched versions found
+---------+---------------+---------------+---------------+
| Package | client | scheduler | workers |
+---------+---------------+---------------+---------------+
| numpy | 1.19.0 | 1.18.1 | 1.18.1 |
| python | 3.7.8.final.0 | 3.7.0.final.0 | 3.7.0.final.0 |
+---------+---------------+---------------+---------------+
warnings.warn(version_module.VersionMismatchWarning(msg[0]["warning"]))
Traceback (most recent call last):
File "<input>", line 7, in <module>
File "/Users/gdesmarais/git/rightsize/rightsize-venv/lib/python3.7/site-packages/distributed/protocol/core.py", line 163, in dumps_msgpack
payload = msgpack.dumps(msg, use_bin_type=True)
File "/Users/gdesmarais/git/rightsize/rightsize-venv/lib/python3.7/site-packages/msgpack/__init__.py", line 35, in packb
return Packer(**kwargs).pack(o)
File "msgpack/_packer.pyx", line 286, in msgpack._cmsgpack.Packer.pack
File "msgpack/_packer.pyx", line 292, in msgpack._cmsgpack.Packer.pack
File "msgpack/_packer.pyx", line 289, in msgpack._cmsgpack.Packer.pack
File "msgpack/_packer.pyx", line 225, in msgpack._cmsgpack.Packer._pack
File "msgpack/_packer.pyx", line 283, in msgpack._cmsgpack.Packer._pack
TypeError: can not serialize 'Variable' object
我缺少包裹吗?我可以提供安装脚本等,但是涉及到一些脚本-客户端是本地python env,调度程序和工作程序是docker。