使用0mq在进程之间建立双向通信的最正确方法是什么?我需要创建几个后台进程,它们将等待来自主进程的命令,执行一些计算并将结果返回给主进程。
答案 0 :(得分:7)
有几种方法可以做到这一点。最直接的方法可能是使用REQ
/ REP
套接字。每个后台进程/ worker都有一个REP
套接字,您可以使用REQ
套接字与它们进行通信:
import zmq
def worker(addr):
context = zmq.Context()
socket = context.socket(zmq.REP)
socket.bind(addr)
while True:
# get message from boss
msg = socket.recv()
# ...do smth
# send back results
socket.send(msg)
if __name__ == '__main__':
# spawn 5 workers
from multiprocessing import Process
for i in range(5):
Process(target=worker, args=('tcp://127.0.0.1:500%d' % i,)).start()
您必须连接到每个工作人员以向他们发送消息,并获取结果:
context = zmq.Context()
socket = context.socket(zmq.REQ)
socket.connect(worker_addr)
socket.send('message')
msg = socket.recv()
另一种方法是使用PUB
/ SUB
向工作人员发送消息,并PUSH
/ PULL
收集结果:
import zmq
def worker(worker_id, publisher_addr, results_addr):
context = zmq.Context()
sub = context.socket(zmq.SUB)
sub.connect(publisher_addr)
sub.setsockopt(zmq.SUBSCRIBE, worker_id)
push = context.socket(zmq.PUSH)
push.connect(results_addr)
while True:
msg = sub.recv_multipart()[1]
# do smth, send off results
push.send_multipart([worker_id, msg])
if __name__ == '__main__':
publisher_addr = 'tcp://127.0.0.1:5000'
results_addr = 'tcp://127.0.0.1:5001'
# launch some workers into space
from multiprocessing import Process
for i in range(5):
Process(target=worker, args=('worker-%d' % i, publisher_addr, results_addr,)).start()
要向特定工作人员广播命令,您可以执行以下操作:
context = zmq.Context()
pub = context.socket(zmq.PUB)
pub.bind(publisher_addr)
# send message to worker-1
pub.send_multipart(['worker-1', 'hello'])
拉入结果:
context = zmq.Context()
pull = context.socket(zmq.PULL)
pull.bind(results_addr)
while True:
worker_id, result = pull.recv_multipart()
print worker_id, result
答案 1 :(得分:3)
考虑使用Request Reply Broker,但将REQ套接字交换为DEALER。经销商不会阻止发送,并会自动为您的员工负载均衡流量。
在图片Client
中,您的main process
和Service A/B/C
将是您的background processes (workers)
。 Main process
应绑定到端点。 Workers
应连接到主进程的端点以接收工作项。
在main process
保留工作项列表并发送时间。如果一段时间内没有答案,则再次重新发送工作项,因为worker
可能已经死亡。