Question

我正在编写一个应用程序来收集UDP消息并每隔1秒处理一次。

应用程序原型如下：

from twisted.internet.protocol import DatagramProtocol
from twisted.internet import reactor
import threading
import time

class UdpListener(DatagramProtocol):

    messages = []

    def datagramReceived(self, data, (host, port)):
        self.messages.append(data)

class Messenger(threading.Thread):

    listener = None

    def __init__(self):
        threading.Thread.__init__(self)

    def run(self):
        while True:
            time.sleep(1)
            recivedMessages = self.listener.messages
            length = len(recivedMessages)
            messagesToProccess = recivedMessages[0:length]
            #doSomethingWithMessages(messagesToProccess)
            del self.listener.messages[0:length]
            print(length)

listener = UdpListener()

messenger = Messenger()
messenger.listener = listener
messenger.start()

reactor.listenUDP(5556, listener)
reactor.run()

我不确定是否可以轻松地从列表中删除起始值（del self.listener.messages [0：length]），而不会有任何传入消息更改列表和应用程序崩溃的风险。

更新 - 带锁的版本

class Messenger(threading.Thread):

listener = None
lock = threading.Lock()

def __init__(self):
    threading.Thread.__init__(self)

def run(self):
    while True:
        time.sleep(1)
        recivedMessages = self.listener.messages
        self.lock.acquire()
        try:
            length = len(recivedMessages)
            messagesToProccess = recivedMessages[0:length]
            del self.listener.messages[0:length]
        except Exception as e:
            raise e
        finally:
            self.lock.release()

        #doSomethingWithMessages(messagesToProccess)
        print(length)

Answer 1

您的代码不是线程安全的，不是。您需要锁定messages。

但是，这里不需要线程。为什么不这样做？

from twisted.internet.protocol import DatagramProtocol
from twisted.internet import reactor

class UdpListener(DatagramProtocol):
    callingLater = False

    messages = []

    def process(self):
        doSomethingWithMessages(self.messages)
        self.messages = []
        self.callingLater = False

    def datagramReceived(self, data, (host, port)):
        self.messages.append(data)
        if not self.callingLater:
            reactor.callLater(1.0, self.process)
            self.callingLater = True

listener = UdpListener()

reactor.listenUDP(5556, listener)
reactor.run()

更新：以下是原始版本如何使用锁定，仅用于教育目的。请注意，这不是那么有效，也更容易出错。编辑：将所有消息逻辑分离到UdpListener，以便使用它的类不需要知道其粘滞的内部细节。

from twisted.internet.protocol import DatagramProtocol
from twisted.internet import reactor
import threading
import time

class UdpListener(DatagramProtocol):
    message_lock = threading.Lock()
    messages = []

    def datagramReceived(self, data, (host, port)):
        with self.message_lock:
            self.messages.append(data)

    def getAndClearMessages(self):
        with self.message_lock:
            res = self.messages
            self.messages = []
        return res

class Messenger(threading.Thread):

    listener = None

    def __init__(self):
        threading.Thread.__init__(self)

    def run(self):
        while True:
            time.sleep(1)
            recivedMessages = self.listener.getAndClearMessages()
            length = len(recivedMessages)
            #doSomethingWithMessages(recivedMessages)
            print(length)

listener = UdpListener()

messenger = Messenger()
messenger.listener = listener
messenger.start()

reactor.listenUDP(5556, listener)
reactor.run()

Answer 2

为什么不用DeferredQueue实现它，这正是出于此目的。如果你想使用线程，你需要格外小心。

以下是DeferredQueue的示例，允许线程化：

class UdpListener(DatagramProtocol):

    def __init__(self)
        self._messages = DeferredQueue()

    def datagramReceived(self, data, (host, port)):
        self._messages.put(message)

    @inlineCallbacks
    def _wait_for_and_process_next_message(self):

        # Get message from queue through a deferred call from the DeferredQueue
        # Here we use @inlineCallbacks, so we assign the result from yield 
        # which is the new message, and will "block" (actually releasing control to Twisted) until a message gets in
        message = yield self._message_queue.get()

        # Do something with your message here, and ensure you catch any exceptions!
        # If your message processing may be long, you may wish to run it in another thread,
        # and because of @inlineCallbacks, this call will "block" until your function finishes.
        # In case you did this, ensure you read the notes below.
        yield threads.deferToThread(my_long_function, message)

        # Schedule an immediate call to this method again in order to process next message
        self.wait_for_and_process_next_message()

    def wait_for_and_process_next_message(self):
        reactor.callLater(0, self._wait_for_and_process_next_message)

    def initialize(self):
        # Call this during your application bootstrapping, so you start processing messages
        self.wait_for_and_process_next_message()

非常重要的是要注意，如果您选择将消息处理推迟到Twisted线程池（使用threads.deferToThread），您的代码将在不同的线程中运行。您可能会响应来自不同线程的消息，并且在Twisted中，协议不是线程安全对象（http://twistedmatrix.com/documents/13.2.0/core/howto/threading.html#auto0）。

对于这种情况，您可以使用reactor.callFromThread()来保护关键资源transport，如下例所示：

def _send_message_critical_section(self, message):
    self.transport.write(message, (self.host, self.port))

def send_message(self, message):
    reactor.callFromThread(self._send_message_critical_section, message)

完成其他更改：

将messages变量重命名为_messages，因为它应被视为完全私有。
在_messages方法中移动__init__()初始化并分配给self._messages，否则消息列表将在所有实例之间共享！我猜你只有一个类的实例，但......（Variables inside and outside of a class __init__() function）

这个python代码线程是否安全（带扭曲的线程）？

2 个答案: