ipython并行负载均衡视图随机失败

时间:2014-02-09 05:10:07

标签: python ipython

这是我的代码:

from IPython.parallel import Client
from sklearn.datasets import load_digits

def mytask(data, labels, id):
   # ...
   pass

engines = Client()
bview = engines.load_balanced_view()
bview.block = False

digits = load_digits()
X, y = digits.data, digits.target

job = bview.apply(mytask, X, y, 1)

while not job.ready(): # line 242
    time.sleep(2)

print job.result

偶尔使用相同的输入我的代码会失败:

Traceback (most recent call last):
  File "task.py", line 242, in <module>
    while not job.ready():
  File "/usr/lib/python2.7/dist-packages/IPython/parallel/client/asyncresult.py", line 111, in ready
    self.wait(0)
  File "/usr/lib/python2.7/dist-packages/IPython/parallel/client/asyncresult.py", line 121, in wait
    self._ready = self._client.wait(self.msg_ids, timeout)
  File "/usr/lib/python2.7/dist-packages/IPython/parallel/client/client.py", line 844, in wait
    self.spin()
  File "/usr/lib/python2.7/dist-packages/IPython/parallel/client/client.py", line 799, in spin
    self._flush_results(self._task_socket)
  File "/usr/lib/python2.7/dist-packages/IPython/parallel/client/client.py", line 692, in _flush_results
    handler(msg)
  File "/usr/lib/python2.7/dist-packages/IPython/parallel/client/client.py", line 657, in _handle_apply_reply
    self.results[msg_id] = util.unserialize_object(msg['buffers'])[0]
  File "/usr/lib/python2.7/dist-packages/IPython/parallel/util.py", line 262, in unserialize_object
    return uncanSequence(map(unserialize, sobj)), bufs
  File "/usr/lib/python2.7/dist-packages/IPython/utils/newserialized.py", line 177, in unserialize
    return UnSerializeIt(serialized).getObject()
  File "/usr/lib/python2.7/dist-packages/IPython/utils/newserialized.py", line 161, in getObject
    result = numpy.frombuffer(buf, dtype = self.serialized.metadata['dtype'])
ValueError: offset must be non-negative and smaller than buffer lenth (0)

这似乎与我的代码无关。我不确定会出现什么问题。

0 个答案:

没有答案