pyzmq经销商未发送给目标员工

时间:2018-07-11 03:42:20

标签: python-3.x multithreading pyzmq

在尝试在pyzmq中获取经销商以向我的目标工作线程发送消息时遇到麻烦。我已经阅读了文档,并且可以肯定我正确地遵循了该示例。

在我的代码中,我有一个ROUTER / DEALER设置。我要实现的目标是让多个客户端连接到ROUTER,并让它们发送将发送到空闲工作线程的请求,然后让工作线程在完成任务后进行回复。我不希望任务长期运行,以阻止其他可能会很快返回任务的客户。

在我的代码中,我将消息发送给空闲的工作人员,但最终消息却发给了忙碌的工作人员。

它在Python 3.5.3,pyzmq 16.0.4和Windows 10上运行。

    # code to setup ROUTER/DEALER protocol

    self._frontend = self._context.socket(zmq.ROUTER)
    self._frontend.bind(self._url)

    url_worker = "inproc://workers"
    self._backend = self._context.socket(zmq.DEALER)
    self._backend.bind(url_worker)

    for i in range(self._worker_threads):
        id = 'w%d' % i
        thread = threading.Thread(target=self.handle_request, args=(url_worker, self._context, id))
        thread.start()

    self.load_balance_requests([w.encode('ascii') for w in self._workers_ready])


# handles request for workers
def handle_request(self, url_worker, context, id):

    self._logger.info('Connect to %s %s' % (url_worker, id))

    socket = context.socket(zmq.REP)
    # used to identify worker
    socket.identity = id.encode('ascii')
    socket.connect(url_worker)
    with self._workers_lock:
        self._workers_ready.append(id)

    while True:
        try:
            #  Wait for next request from client
            self._logger.info('Wait for message')
            address, empty, zmsg = socket.recv_multipart()
            self._logger.info('Request received')

def load_balance_requests(self, workers):
    poller = zmq.Poller()
    poller.register(self._backend, zmq.POLLIN)
    poller.register(self._frontend, zmq.POLLIN)

    while True:
        sockets = dict(poller.poll())

        if self._backend in sockets:
            # Handle worker activity on the backend
            request = self._backend.recv_multipart()
            worker, empty, client = request[:3]
            if not workers:
                # Poll for clients now that a worker is available
                poller.register(self._frontend, zmq.POLLIN)
            workers.append(worker)
            empty, reply = request[3:]
            self._logger.info('Sending to %s from %s' % (str(client), str(worker)))
            self._frontend.send_multipart([client, b"", reply])

        if self._frontend in sockets:
            # Get next client request, route to last-used worker
            client, empty, request = self._frontend.recv_multipart()
            worker = workers.pop(0)
            self._logger.info('Sending to %s from %s' % (str(worker), str(client)))
            self._backend.send_multipart([worker, b"", client, b"", request])
            if not workers:
                # Don't poll clients if no workers are available
                poller.unregister(self._frontend)

    # Clean up
    self._backend.close()
    self._frontend.close()
    self._context.term()    

这是我的日志输出。我在线程31448上有工作程序w0,在线程14180上有工作程序w1,在线程30932上有工作程序w2。发送给w0的请求是一个长期运行的任务。接下来的2个请求转到w1和w2。到目前为止,一切都很好。但是随后又有另一个请求发送到w1。但是,请求将阻塞,并最终显示在线程31448上,该线程为工作线程w0。它最终会通过,但只有在第一个w0任务完成之后。

不知道我在做什么错了。

2018-07-11 11:24:25,747 - myprogram INFO 31964 31448 - Connect to inproc://workers w0
2018-07-11 11:24:25,747 - myprogram INFO 31964 14180 - Connect to inproc://workers w1
2018-07-11 11:24:25,759 - myprogram INFO 31964 30932 - Connect to inproc://workers w2
2018-07-11 11:25:08,156 - myprogram INFO 31964 2036 - Sending to b'w0' from b'\x00\x80\x00\x00)'
2018-07-11 11:25:08,156 - myprogram INFO 31964 31448 - Request received
2018-07-11 11:25:12,801 - myprogram INFO 31964 2036 - Sending to b'w1' from b'\x00\x80\x00\x00*'
2018-07-11 11:25:12,801 - myprogram INFO 31964 14180 - Request to received
2018-07-11 11:25:13,635 - myprogram INFO 31964 2036 - Sending to b'\x00\x80\x00\x00*' from b'w1'
2018-07-11 11:25:15,346 - myprogram INFO 31964 2036 - Sending to b'w2' from b'\x00\x80\x00\x00+'
2018-07-11 11:25:15,351 - myprogram INFO 31964 30932 - Request received
2018-07-11 11:25:20,239 - myprogram INFO 31964 2036 - Sending to b'\x00\x80\x00\x00+' from b'w2'
2018-07-11 11:25:20,542 - myprogram INFO 31964 2036 - Sending to b'w1' from b'\x00\x80\x00\x00,'
2018-07-11 11:27:05,911 - myprogram INFO 31964 2036 - Sending to b'\x00\x80\x00\x00)' from b'w0'
2018-07-11 11:27:05,912 - myprogram INFO 31964 31448 - Request received
2018-07-11 11:27:06,468 - myprogram INFO 31964 2036 - Sending to b'\x00\x80\x00\x00,' from b'w1'

0 个答案:

没有答案