集正在被修改,然后似乎神奇地恢复

时间:2014-03-13 22:28:58

标签: python multithreading set sync

我试图在不可靠的频道上建立一个可靠传递的频道(这是一个练习,不可靠的频道明确地丢弃了一些频道)。我有一个包含acks对的(address, sequence_number)集。当通过频道收到确认时,它会被添加到acks集并通知条件变量:

msg, addr = self.unreliable_channel.recv()
if isinstance(msg, Ack):
    with self.acks_cond:
        self.acks.add((addr, msg.ack))
        print "{} got an ack: {} ({})".format(self.port, self.acks, hex(id(self.acks)))
        self.acks_cond.notify()

另一个线程正在侦听条件变量并检查线程中的确认:

with self.acks_cond:
    max_wait = 2 * self.unreliable_channel.delay_avg
    start = time.time()
    while not ((addr, msg.seq) in self.acks) and (time.time() - start < max_wait):
        print "{}: self.acks is {} ({})".format(self.port, self.acks, hex(id(self.acks)))
        self.acks_cond.wait(0.1)
    print "{} waited for ack of {} from {}: {} ({})".format(self.port, msg.seq, addr, self.acks, hex(id(self.acks)))
    if (addr, msg.seq) in self.acks:
        print '!' * 10000
        # self.acks.remove((addr, msg.seq))
        return

然而,第二个片段似乎无法看到修改过的集合:

10000 got an ack: set([(('192.168.1.7', 10001), 1), 'toplel']) (0x7f40a944ced0)
10000: self.acks is set(['toplel']) (0x7f40a944ced0)

'toplel'是我投入集合中的一个字符串,以确保它不会以某种方式被清空)

任何人都知道会弄乱这件事吗?

下面的代码转储:(尝试制作一个SSCCE,但似乎无法重现这种行为 - 如果我得到一个,我会尝试更多)。

import threading
import time
from collections import defaultdict, namedtuple

Message = namedtuple('Message', ['seq', 'data'])
Ack = namedtuple('Ack', ['ack'])

class ReliableChannel:
    '''Building on top of UnreliableChannel, this channel supports
    guarenteed eventual delivery, FIFO delivery on a per-destination
    basis, and no duplicated delivery.'''

    def __init__(self, unreliable_channel):
        self.unreliable_channel = unreliable_channel

        # a set of (addr, seq) pairs for which we've recieved acks
        print "INITIALIZING THE TOPLEL"
        self.acks = set(["toplel"])
        self.acks_cond = threading.Condition()

        self.seq = defaultdict(int)

        self.port = self.unreliable_channel.sock.getsockname()[1]
        print 'self.port: {}'.format(self.port)

        # leftoff: the thread we started can't seem to modify self.acks so that unicast can see it
        self.listener = threading.Thread(target=self.listen)
        self.listener.start()

    def listen(self):
        while True:
            msg, addr = self.unreliable_channel.recv()
            if isinstance(msg, Ack):
                with self.acks_cond:
                    self.acks.add((addr, msg.ack))
                    print "{} got an ack: {} ({})".format(self.port, self.acks, hex(id(self.acks)))
                    self.acks_cond.notify()
            else:
                ack = Ack(msg.seq)
                self.unreliable_channel.unicast(ack, addr)
                print '{} Got message {} and sent {} back to {}'.format(self.port, msg, ack, addr)

    def unicast(self, msg, addr):
        self.seq[addr] += 1  # get the sequence number for this message
        msg = Message(self.seq[addr], msg)
        print '{} Trying to send message {} to {}'.format(self.port, msg, addr)
        while True:
            # send a message
            self.unreliable_channel.unicast(msg, addr)

            # wait for an ack with a timeout
            with self.acks_cond:
                max_wait = 2 * self.unreliable_channel.delay_avg
                start = time.time()
                while not ((addr, msg.seq) in self.acks) and (time.time() - start < max_wait):
                    print "{}: self.acks is {} ({})".format(self.port, self.acks, hex(id(self.acks)))
                    self.acks_cond.wait(0.1)
                print "{} waited for ack of {} from {}: {} ({})".format(self.port, msg.seq, addr, self.acks, hex(id(self.acks)))
                if (addr, msg.seq) in self.acks:
                    print '!' * 10000
                    # self.acks.remove((addr, msg.seq))
                    return

编辑:在做了一些更多的搞乱之后,看起来这些集合有时候会发生变化&#34;一段时间后在两个线程中。根据我在listen中的位置,对列表的修改有时需要,但在此之后,似乎每个线程都在使用它自己的集合副本(即,我尝试在集合中添加内容)两个线程)。

1 个答案:

答案 0 :(得分:0)

自我回答:知道这是相当愚蠢的事情。上面的__init__方法启动了一个线程,它正在__init__子类的multiprocessing.Process方法中运行。这意味着正在运行listen的线程正在一个进程中启动,而运行unicast的线程正在另一个进程中运行,并且我猜测该副本的副本。

应该早点意识到这一点,但id s相同的事实让我失望了 - 我想我认为分叉进程会获得自己的虚拟地址,所以它的可能性很大同样的人会很低。没想到假设它会重新使用相同的虚拟地址。

tl; dr:id(x) == id(y)并不意味着xy是同一个对象,如果您正在搞乱多处理。