如何通过SCOOP Python发布工作者和经纪人?

时间:2016-02-17 22:19:10

标签: python distributed-computing python-multiprocessing

遵循使用SCOOP跨不同虚拟机进行多处理的简单Python代码,似乎根本不起作用:

from __future__ import print_function
from scoop import futures

def helloWorld(value):
    return "Hello World from Future #{0}".format(value)

if __name__ == "__main__":
    returnValues = list(futures.map(helloWorld, range(16)))
    print("\n".join(returnValues))

仅供参考 - 我有相同的虚拟机(192.168.0.13和192.168.0.12),并且已经设置了没有密码的SSH隧道。

我使用以下命令启动程序:

user_a@vm12:~$ python -m scoop --hosts vm13 vm13 -vv --verbose --external-hostname 192.168.0.13 --path /home/user_a ./pycode/hw-01.py

我能看到的唯一输出是:

[2016-02-18 09:09:25,222] launcher  INFO    SCOOP 0.7 1.1 on linux2 using Python 2.7.6 (default, Jun 22 2015, 18:00:18) [GCC 4.8.2], API: 1013
[2016-02-18 09:09:25,223] launcher  INFO    Deploying 2 worker(s) over 1 host(s).
[2016-02-18 09:09:25,223] launcher  DEBUG   Using hostname/ip: "192.168.0.13" as external broker reference.
[2016-02-18 09:09:25,223] launcher  DEBUG   The python executable to execute the program with is: /usr/bin/python.
[2016-02-18 09:09:25,223] launcher  INFO    Worker distribution: 
[2016-02-18 09:09:25,223] launcher  INFO       vm13:    1 + origin
[2016-02-18 09:09:25,224] brokerLaunch DEBUG   Launching remote broker: ssh -x -n -oStrictHostKeyChecking=no vm13 /usr/bin/python -m scoop.broker.__main__ --echoGroup --echoPorts --backend ZMQ 
[2016-02-18 09:09:29,252] brokerLaunch DEBUG   Foreign broker launched on ports 44950, 55874 of host vm13.
[2016-02-18 09:09:29,253] launcher  DEBUG   Initialising remote worker 2 [vm13].
[2016-02-18 09:09:29,253] launcher  DEBUG   Initialising remote origin worker 1 [vm13].
[2016-02-18 09:09:29,254] launcher  DEBUG   vm13: Launching '(/usr/bin/python -m scoop.bootstrap.__main__ --echoGroup  --size 2 --workingDirectory /home/user_a --brokerHostname 192.168.0.13 --externalBrokerHostname 192.168.0.13 --taskPort 44950 --metaPort 55874 --backend=ZMQ -v ./pycode/hw-01.py & ) && (/usr/bin/python -m scoop.bootstrap.__main__ --size 2 --workingDirectory /home/user_a --brokerHostname 192.168.0.13 --externalBrokerHostname 192.168.0.13 --taskPort 44950 --metaPort 55874 --origin --backend=ZMQ -v ./pycode/hw-01.py)'
[2016-02-18 09:09:29,536] __main__  INFO    Worker(s) launched using /bin/bash

这很奇怪,因为如果我只在一台机器上运行它,一切都很顺利:

python -m scoop ./pycode/hw-01.py 
[2016-02-18 09:12:52,739] launcher  INFO    SCOOP 0.7 1.1 on linux2 using Python 2.7.6 (default, Jun 22 2015, 18:00:18) [GCC 4.8.2], API: 1013
[2016-02-18 09:12:52,739] launcher  INFO    Deploying 2 worker(s) over 1 host(s).
[2016-02-18 09:12:52,739] launcher  INFO    Worker distribution: 
[2016-02-18 09:12:52,740] launcher  INFO       127.0.0.1:   1 + origin
Hello World from Future #0
Hello World from Future #1
Hello World from Future #2
Hello World from Future #3
Hello World from Future #4
Hello World from Future #5
Hello World from Future #6
Hello World from Future #7
Hello World from Future #8
Hello World from Future #9
Hello World from Future #10
Hello World from Future #11
Hello World from Future #12
Hello World from Future #13
Hello World from Future #14
Hello World from Future #15
[2016-02-18 09:12:53,174] launcher  (127.0.0.1:34747) INFO    Root process is done.
[2016-02-18 09:12:53,175] launcher  (127.0.0.1:34747) INFO    Finished cleaning spawned subprocesses.

非常感谢任何想法或想法。

0 个答案:

没有答案