我正在与PyZMQ合作,我看起来似乎是一个相当特殊的问题。我有两个包装用于通信的套接字,MZLSubscriber
和MZLRequester
。有一个类包含它们MZLLink
。对于其中的每一项,我还有测试MZLSubscriberTest
,MZLRequesterTest
和MZLinkTest
。订阅者和请求者的测试工作正常,但MZLinkTest
不会收到任何订阅者消息。
下面是相对代码,它们是3个类的构造函数,run()
的{{1}}以及MZLSubscriber
和MZLink
的测试
MZLSubscriber
构造函数:
MZLink
# Host information
self.host = host
self.requestPort = requestPort
self.subscriberPort = subscriberPort
# Set up zmq context
self.zmq_context = zmq.Context()
# Set up subscriber and replier
self.subscriber = MZLSubscriber(self.zmq_context, self.host, self.subscriberPort)
self.requester = MZLRequester(self.zmq_context, self.host, self.requestPort)
# Start subscriber
self.subscriber.start()
测试:
MZLink
# Constants
HOST = "localhost"
REQ_PORT = 5555
SUB_PORT = 5556
# Create Link
link = MIDASZMQLink(HOST, REQ_PORT, SUB_PORT)
link.close()
构造函数:
MZLRequester
# Initialize class member variables
self.zmq_context = zmq_context
self.host = host
self.port = port
# Set up reply socket
self.socket = self.zmq_context.socket(zmq.REQ)
# Connect socket
self.socket.connect("tcp://{0}:{1}".format(self.host, self.port))
构造函数:
MZLSubscriber
# Initialize parent process
Process.__init__(self)
# Store zmq context and connection host/port
self.zmq_context = zmq_context
self.host = host
self.port = port
# Sockets. Don't set them up here because sockets are not thread safe
self.socket = None
# Queue to store data in
# TODO: Make queue not overflow if events come in too quickly
self.queue = Queue()
:
MZLSubscriber.run()
# Parent call
Process.run(self)
# Set up subscriber socket in this thread
self.socket = self.zmq_context.socket(zmq.SUB)
self.socket.setsockopt_string(zmq.SUBSCRIBE, unicode())
# Connect socket
self.socket.connect("tcp://{0}:{1}".format(self.host, self.port))
# While the thread is alive, poll for data to put into queue
# Calling MZLSubscriber.stop() will automatically change this
while self.is_alive():
datum = self.socket.recv()
self.queue.put(datum)
# Disconnect and close socket.
#FIXME: Doesn't get here because terminate() immediately stops the process
self.socket.disconnect("tcp://{0}:{1}".format(self.host, self.port))
self.socket.close()
测试:
MZLSubscriber
订阅者线程似乎在# Host information
HOST = "localhost"
SUBSCRIBER_PORT = "5556"
# Set up zmq context
zmq_context = zmq.Context()
# Set up subscriber
subscriber = MZLSubscriber(zmq_context, HOST, SUBSCRIBER_PORT)
# Start subscriber
subscriber.start()
# Stop and join subscriber
subscriber.close()
subscriber.join()
处阻塞,这让我觉得它可能是套接字创建的一些问题。但是,它似乎只在与订户一起工作时才起作用。请求者似乎在两种情况下都有效。此外,只需评论处理datum = self.socket.recv()
的两行,即可顺利完成所有工作。
我为代码墙道歉,但我甚至无法缩小此问题的代码。当我这样做时,我将删除不相关的代码。处理传入数据的测试代码已被删除。
作为一点澄清,我使用Python 2.7和PyZMQ 14.3.1。
更新:似乎在主线程中运行requester
而不是创建另一个MZLSubscriber
会产生预期的结果,所以看起来这可能是某种线程安全。据我所知,zmq上下文是线程安全的,但套接字不是。我认为这不会引起问题,因为我明确确定每个线程都有一个套接字。
更新2 :如果设置Process
中的套接字的呼叫从MZLSubscriber
移至run()
,则套接字似乎会收到一小部分已发布的消息,但确实有错误:
__init__
我已经通过在Process MZLSubscriber-1:
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/Users/user/Repos/midas-client/client/midasclient/mzlSubscriber.py", line 45, in run
datum = self.socket.recv()
File "socket.pyx", line 628, in zmq.backend.cython.socket.Socket.recv (zmq/backend/cython/socket.c:5616)
File "socket.pyx", line 662, in zmq.backend.cython.socket.Socket.recv (zmq/backend/cython/socket.c:5436)
File "socket.pyx", line 139, in zmq.backend.cython.socket._recv_copy (zmq/backend/cython/socket.c:1771)
File "checkrc.pxd", line 21, in zmq.backend.cython.checkrc._check_rc (zmq/backend/cython/socket.c:6032)
ZMQError: Interrupted system call
中创建新的zmq.Context
来解决此问题,但我觉得如果zmq上下文是线程安全的,则不应该这样做。
答案 0 :(得分:0)
似乎我的问题是在不同进程上使用多个zmq上下文。虽然PyZMQ文档声明zmq上下文是线程安全的,但我只能假设它意味着Python线程而不是进程。这很令人困惑,因为在C中,zmq上下文是线程安全的,尽管以类似于Python multiprocessing.Process
的方式运行。
通过为每个进程创建zmq上下文来解决该问题。