Question

我在dask下运行一些扩展测试。我从中创建一个LocalCluster和一个客户端，进行一些处理，然后关闭客户端并关闭LocalCluster。在第一次测试中，端口8787按预期用于散景。但是在后续测试中，端口号不是8787，而是一些随机数。这是一个说明问题的脚本：

from distributed import Client
import time

if __name__ == '__main__':

    max_n_workers = 8
    print('Maximum number of workers is ', max_n_workers)

    n_workers = max_n_workers
    while n_workers > 1:
        c = Client(n_workers=n_workers)
        print(c)
        addr = c.scheduler_info()['address']
        services = c.scheduler_info()['services']
        if 'bokeh' in services.keys():
            bokeh_addr = 'http:%s:%s' % (addr.split(':')[1], services['bokeh'])
            print('Diagnostic pages available on port %s' % bokeh_addr)

        c.shutdown()

        n_workers = n_workers // 2
        time.sleep(10)

exit()

示例输出

Maximum number of workers is  8
<Client: scheduler='tcp://127.0.0.1:41049' processes=8 cores=16>
Diagnostic pages available on port http://127.0.0.1:8787
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed
<Client: scheduler='tcp://127.0.0.1:34152' processes=4 cores=16>
Diagnostic pages available on port http://127.0.0.1:39621
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed
<Client: scheduler='tcp://127.0.0.1:37583' processes=2 cores=16>
Diagnostic pages available on port http://127.0.0.1:45183
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed

Answer 1

当您关闭LocalCluster时，它会关闭所有服务。这会将所有以前使用的端口返回给操作系统。当您启动新的LocalCluster时，它会再次向操作系统询问这些端口。如果散景端口尚不可访问，那么它将选择一个新的随机端口而不是失败。

在您的情况下，我怀疑操作系统尚未将默认端口注册为打开。我怀疑，如果你给操作系统一些时间，事情就会清理。

dask散景端口不能重复使用

1 个答案: