Dask分布式客户端需要很长时间才能在Jupyter Lab中进行初始化

时间:2018-11-02 14:03:56

标签: python-3.x dask dask-distributed jupyter-lab

尝试在Jupyter实验室中使用本地群集初始化客户端,但挂起。 python 3.5和jupyter lab 0.35会发生这种情况。

import dask.dataframe as dd
from dask import delayed
from distributed import Client
from distributed import LocalCluster
import pandas as pd
import numpy as np
import json

cluster = LocalCluster()
client = Cluster(cluster)
client

工具的版本:

Python 3.5.2 (default, Nov 23 2017, 16:37:01) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 

jupyter实验室版本是

pip list |grep jupyter
jupyter            1.0.0  
jupyter-client     5.2.3  
jupyter-console    6.0.0  
jupyter-core       4.4.0  
jupyterlab         0.35.3 
jupyterlab-server  0.2.0  

早期版本

pip list |grep dask
dask               0.20.0 

怎么了?

编辑:尝试初始化客户端时终端上没有异常 中断内核具有以下例外情况: 内部jupyter:

distributed.nanny - ERROR - Failed to restart worker after its process exited
Traceback (most recent call last):
  File "/home/avlach/virtualenvs/staistics3/lib/python3.5/site-packages/distributed/nanny.py", line 291, in _on_exit
    yield self.instantiate()
  File "/home/avlach/virtualenvs/staistics3/lib/python3.5/site-packages/tornado/gen.py", line 1133, in run
    value = future.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/home/avlach/virtualenvs/staistics3/lib/python3.5/site-packages/tornado/gen.py", line 1141, in run
    yielded = self.gen.throw(*exc_info)
  File "/home/avlach/virtualenvs/staistics3/lib/python3.5/site-packages/distributed/nanny.py", line 226, in instantiate
    self.process.start()
  File "/home/avlach/virtualenvs/staistics3/lib/python3.5/site-packages/tornado/gen.py", line 1133, in run
    value = future.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
raise self._exception
  File "/home/avlach/virtualenvs/staistics3/lib/python3.5/site-packages/tornado/gen.py", line 1141, in run
    yielded = self.gen.throw(*exc_info)
  File "/home/avlach/virtualenvs/staistics3/lib/python3.5/site-packages/distributed/nanny.py", line 370, in start
    yield self.process.start()
  File "/home/avlach/virtualenvs/staistics3/lib/python3.5/site-packages/tornado/gen.py", line 1133, in run
    value = future.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/home/avlach/virtualenvs/staistics3/lib/python3.5/site-packages/distributed/process.py", line 35, in _call_and_set_future
    res = func(*args, **kwargs)
  File "/home/avlach/virtualenvs/staistics3/lib/python3.5/site-packages/distributed/process.py", line 184, in _start
    process.start()
  File "/usr/lib/python3.5/multiprocessing/process.py", line 105, in start
    self._popen = self._Popen(self)
  File "/usr/lib/python3.5/multiprocessing/context.py", line 281, in _Popen
    return Popen(process_obj)
      File "/usr/lib/python3.5/multiprocessing/popen_forkserver.py", line 36, in __init__
        super().__init__(process_obj)
      File "/usr/lib/python3.5/multiprocessing/popen_fork.py", line 20, in __init__
        self._launch(process_obj)
      File "/usr/lib/python3.5/multiprocessing/popen_forkserver.py", line 52, in _launch
        self.sentinel, w = forkserver.connect_to_new_process(self._fds)
      File "/usr/lib/python3.5/multiprocessing/forkserver.py", line 66, in connect_to_new_process
    client.connect(self._forkserver_address)
    ConnectionRefusedError: [Errno 111] Connection refused

并在终端上

Kernel interrupted: 1e7da6ed-2137-41ce-818e-3484c0a659cc
fd_event_list = self._epoll.poll(timeout, max_ev)
KeyboardInterrupt

0 个答案:

没有答案