当LocalCluster使用线程而不是进程时,变量无法反序列化

时间:2018-06-27 14:47:10

标签: dask-distributed

我正在尝试正常关闭任务 似乎在将LocalCluster与 线程而不是进程 这是我使用的代码

<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
<div class="btn-group btn-group-toggle" data-toggle="buttons">
<label class="btn btn-secondary active">
  <input type="radio" name="options" class="option" autocomplete="off" value="hello 1" checked> Active
</label>
<label class="btn btn-secondary">
  <input type="radio" name="options" class="option" autocomplete="off" value="hi 2" /> Radio
</label>
<label class="btn btn-secondary">
  <input type="radio" name="options" class="option" autocomplete="off" value="foo 3" /> Radio
</label>

<div class="showcontent"></div>

这将失败,并显示以下错误

from distributed import Client, LocalCluster
cluster = LocalCluster(processes=False)
client = Client(cluster)

from dask.distributed import Variable
import time 

stop = Variable()
stop.set(False)

def my_task():
    while True:
        if stop.get():
            return

        else:
            print "hello"
            time.sleep(10)
    print "main loop exit"

future = client.submit(my_task)

# wait
stop.set(True)

如果集群的定义是这样的

In [3]: distributed.worker - WARNING - Could not deserialize task
Traceback (most recent call last):
  File "/anaconda/lib/python2.7/site-packages/distributed/worker.py", line 1266, in add_task
    self.tasks[key] = _deserialize(function, args, kwargs, task)
  File "/anaconda/lib/python2.7/site-packages/distributed/worker.py", line 642, in _deserialize
    function = pickle.loads(function)
  File "/anaconda/lib/python2.7/site-packages/distributed/protocol/pickle.py", line 59, in loads
    return pickle.loads(x)
  File "/anaconda/lib/python2.7/site-packages/distributed/variable.py", line 212, in __setstate__
    client = Client(address, set_as_default=False)
  File "/anaconda/lib/python2.7/site-packages/distributed/client.py", line 578, in __init__
    self.start(timeout=timeout)
  File "/anaconda/lib/python2.7/site-packages/distributed/client.py", line 698, in start
    sync(self.loop, self._start, **kwargs)
  File "/anaconda/lib/python2.7/site-packages/distributed/utils.py", line 253, in sync
    six.reraise(*error[0])
  File "/anaconda/lib/python2.7/site-packages/distributed/utils.py", line 238, in f
    result[0] = yield make_coro()
  File "/anaconda/lib/python2.7/site-packages/tornado/gen.py", line 1055, in run
    value = future.result()
  File "/anaconda/lib/python2.7/site-packages/tornado/concurrent.py", line 238, in result
    raise_exc_info(self._exc_info)
  File "/anaconda/lib/python2.7/site-packages/tornado/gen.py", line 1063, in run
    yielded = self.gen.throw(*exc_info)
  File "/anaconda/lib/python2.7/site-packages/distributed/client.py", line 778, in _start
    yield self._ensure_connected(timeout=timeout)
  File "/anaconda/lib/python2.7/site-packages/tornado/gen.py", line 1055, in run
    value = future.result()
  File "/anaconda/lib/python2.7/site-packages/tornado/concurrent.py", line 238, in result
    raise_exc_info(self._exc_info)
  File "/anaconda/lib/python2.7/site-packages/tornado/gen.py", line 1063, in run
    yielded = self.gen.throw(*exc_info)
  File "/anaconda/lib/python2.7/site-packages/distributed/client.py", line 815, in _ensure_connected
    connection_args=self.connection_args)
  File "/anaconda/lib/python2.7/site-packages/tornado/gen.py", line 1055, in run
    value = future.result()
  File "/anaconda/lib/python2.7/site-packages/tornado/concurrent.py", line 238, in result
    raise_exc_info(self._exc_info)
  File "/anaconda/lib/python2.7/site-packages/tornado/gen.py", line 1063, in run
    yielded = self.gen.throw(*exc_info)
  File "/anaconda/lib/python2.7/site-packages/distributed/comm/core.py", line 196, in connect
    _raise(error)
  File "/anaconda/lib/python2.7/site-packages/distributed/comm/core.py", line 179, in _raise
    raise IOError(msg)
IOError: Timed out trying to connect to 'inproc://10.196.73.29/81901/1' after 10 s: connect() didn't finish in time

尝试连接到'inproc://10.196.73.29/81901/1'超时 知道为什么吗?

0 个答案:

没有答案