在尝试在pyzmq中获取经销商以向我的目标工作线程发送消息时遇到麻烦。我已经阅读了文档,并且可以肯定我正确地遵循了该示例。
在我的代码中,我有一个ROUTER / DEALER设置。我要实现的目标是让多个客户端连接到ROUTER,并让它们发送将发送到空闲工作线程的请求,然后让工作线程在完成任务后进行回复。我不希望任务长期运行,以阻止其他可能会很快返回任务的客户。
在我的代码中,我将消息发送给空闲的工作人员,但最终消息却发给了忙碌的工作人员。
它在Python 3.5.3,pyzmq 16.0.4和Windows 10上运行。
# code to setup ROUTER/DEALER protocol
self._frontend = self._context.socket(zmq.ROUTER)
self._frontend.bind(self._url)
url_worker = "inproc://workers"
self._backend = self._context.socket(zmq.DEALER)
self._backend.bind(url_worker)
for i in range(self._worker_threads):
id = 'w%d' % i
thread = threading.Thread(target=self.handle_request, args=(url_worker, self._context, id))
thread.start()
self.load_balance_requests([w.encode('ascii') for w in self._workers_ready])
# handles request for workers
def handle_request(self, url_worker, context, id):
self._logger.info('Connect to %s %s' % (url_worker, id))
socket = context.socket(zmq.REP)
# used to identify worker
socket.identity = id.encode('ascii')
socket.connect(url_worker)
with self._workers_lock:
self._workers_ready.append(id)
while True:
try:
# Wait for next request from client
self._logger.info('Wait for message')
address, empty, zmsg = socket.recv_multipart()
self._logger.info('Request received')
def load_balance_requests(self, workers):
poller = zmq.Poller()
poller.register(self._backend, zmq.POLLIN)
poller.register(self._frontend, zmq.POLLIN)
while True:
sockets = dict(poller.poll())
if self._backend in sockets:
# Handle worker activity on the backend
request = self._backend.recv_multipart()
worker, empty, client = request[:3]
if not workers:
# Poll for clients now that a worker is available
poller.register(self._frontend, zmq.POLLIN)
workers.append(worker)
empty, reply = request[3:]
self._logger.info('Sending to %s from %s' % (str(client), str(worker)))
self._frontend.send_multipart([client, b"", reply])
if self._frontend in sockets:
# Get next client request, route to last-used worker
client, empty, request = self._frontend.recv_multipart()
worker = workers.pop(0)
self._logger.info('Sending to %s from %s' % (str(worker), str(client)))
self._backend.send_multipart([worker, b"", client, b"", request])
if not workers:
# Don't poll clients if no workers are available
poller.unregister(self._frontend)
# Clean up
self._backend.close()
self._frontend.close()
self._context.term()
这是我的日志输出。我在线程31448上有工作程序w0,在线程14180上有工作程序w1,在线程30932上有工作程序w2。发送给w0的请求是一个长期运行的任务。接下来的2个请求转到w1和w2。到目前为止,一切都很好。但是随后又有另一个请求发送到w1。但是,请求将阻塞,并最终显示在线程31448上,该线程为工作线程w0。它最终会通过,但只有在第一个w0任务完成之后。
不知道我在做什么错了。
2018-07-11 11:24:25,747 - myprogram INFO 31964 31448 - Connect to inproc://workers w0
2018-07-11 11:24:25,747 - myprogram INFO 31964 14180 - Connect to inproc://workers w1
2018-07-11 11:24:25,759 - myprogram INFO 31964 30932 - Connect to inproc://workers w2
2018-07-11 11:25:08,156 - myprogram INFO 31964 2036 - Sending to b'w0' from b'\x00\x80\x00\x00)'
2018-07-11 11:25:08,156 - myprogram INFO 31964 31448 - Request received
2018-07-11 11:25:12,801 - myprogram INFO 31964 2036 - Sending to b'w1' from b'\x00\x80\x00\x00*'
2018-07-11 11:25:12,801 - myprogram INFO 31964 14180 - Request to received
2018-07-11 11:25:13,635 - myprogram INFO 31964 2036 - Sending to b'\x00\x80\x00\x00*' from b'w1'
2018-07-11 11:25:15,346 - myprogram INFO 31964 2036 - Sending to b'w2' from b'\x00\x80\x00\x00+'
2018-07-11 11:25:15,351 - myprogram INFO 31964 30932 - Request received
2018-07-11 11:25:20,239 - myprogram INFO 31964 2036 - Sending to b'\x00\x80\x00\x00+' from b'w2'
2018-07-11 11:25:20,542 - myprogram INFO 31964 2036 - Sending to b'w1' from b'\x00\x80\x00\x00,'
2018-07-11 11:27:05,911 - myprogram INFO 31964 2036 - Sending to b'\x00\x80\x00\x00)' from b'w0'
2018-07-11 11:27:05,912 - myprogram INFO 31964 31448 - Request received
2018-07-11 11:27:06,468 - myprogram INFO 31964 2036 - Sending to b'\x00\x80\x00\x00,' from b'w1'