set is being modified and then seems to magically revert

问题

I'm trying to build a channel with reliable delivery on top of an unreliable channel (it's an exercise, the unreliable channel explicitly drops some of its packets). I have an acks set that contains (address, sequence_number) pairs. When an ack is received over the channel it's added to the acks set and a condition variable is notified:

msg, addr = self.unreliable_channel.recv()
if isinstance(msg, Ack):
    with self.acks_cond:
        self.acks.add((addr, msg.ack))
        print "{} got an ack: {} ({})".format(self.port, self.acks, hex(id(self.acks)))
        self.acks_cond.notify()

A different thread is listening on the condition variable and checking for the acks in the thread:

with self.acks_cond:
    max_wait = 2 * self.unreliable_channel.delay_avg
    start = time.time()
    while not ((addr, msg.seq) in self.acks) and (time.time() - start < max_wait):
        print "{}: self.acks is {} ({})".format(self.port, self.acks, hex(id(self.acks)))
        self.acks_cond.wait(0.1)
    print "{} waited for ack of {} from {}: {} ({})".format(self.port, msg.seq, addr, self.acks, hex(id(self.acks)))
    if (addr, msg.seq) in self.acks:
        print '!' * 10000
        # self.acks.remove((addr, msg.seq))
        return

However, the second snippet can't seem to see the modified set:

10000 got an ack: set([(('192.168.1.7', 10001), 1), 'toplel']) (0x7f40a944ced0)
10000: self.acks is set(['toplel']) (0x7f40a944ced0)

('toplel' is a string I threw in the set to make sure it wasn't just somehow being emptied)

Anyone have an idea of what would be messing this up?

Code dump below: (tried making a SSCCE but couldn't seem to reproduce the behavior -- I'll try some more though if I get one).

import threading
import time
from collections import defaultdict, namedtuple

Message = namedtuple('Message', ['seq', 'data'])
Ack = namedtuple('Ack', ['ack'])

class ReliableChannel:
    '''Building on top of UnreliableChannel, this channel supports
    guarenteed eventual delivery, FIFO delivery on a per-destination
    basis, and no duplicated delivery.'''

    def __init__(self, unreliable_channel):
        self.unreliable_channel = unreliable_channel

        # a set of (addr, seq) pairs for which we've recieved acks
        print "INITIALIZING THE TOPLEL"
        self.acks = set(["toplel"])
        self.acks_cond = threading.Condition()

        self.seq = defaultdict(int)

        self.port = self.unreliable_channel.sock.getsockname()[1]
        print 'self.port: {}'.format(self.port)

        # leftoff: the thread we started can't seem to modify self.acks so that unicast can see it
        self.listener = threading.Thread(target=self.listen)
        self.listener.start()

    def listen(self):
        while True:
            msg, addr = self.unreliable_channel.recv()
            if isinstance(msg, Ack):
                with self.acks_cond:
                    self.acks.add((addr, msg.ack))
                    print "{} got an ack: {} ({})".format(self.port, self.acks, hex(id(self.acks)))
                    self.acks_cond.notify()
            else:
                ack = Ack(msg.seq)
                self.unreliable_channel.unicast(ack, addr)
                print '{} Got message {} and sent {} back to {}'.format(self.port, msg, ack, addr)

    def unicast(self, msg, addr):
        self.seq[addr] += 1  # get the sequence number for this message
        msg = Message(self.seq[addr], msg)
        print '{} Trying to send message {} to {}'.format(self.port, msg, addr)
        while True:
            # send a message
            self.unreliable_channel.unicast(msg, addr)

            # wait for an ack with a timeout
            with self.acks_cond:
                max_wait = 2 * self.unreliable_channel.delay_avg
                start = time.time()
                while not ((addr, msg.seq) in self.acks) and (time.time() - start < max_wait):
                    print "{}: self.acks is {} ({})".format(self.port, self.acks, hex(id(self.acks)))
                    self.acks_cond.wait(0.1)
                print "{} waited for ack of {} from {}: {} ({})".format(self.port, msg.seq, addr, self.acks, hex(id(self.acks)))
                if (addr, msg.seq) in self.acks:
                    print '!' * 10000
                    # self.acks.remove((addr, msg.seq))
                    return

edit: After doing some more messing around, it seems like the sets sometimes "diverge" in the two threads after a while. Depending on where I do it in listen, the modifications to the list sometimes take, but after that, it seems like each thread is working with its own copy of the set (ie, I tried adding stuff to the set in both threads).

回答1:

Self-answer: Knew it be something fairly stupid. The __init__ method above launches a thread, and it was being run in the __init__ method of a multiprocessing.Process subclass. Meaning the thread running listen was being launched in one process, while the thread running unicast was being run in a separate one, with a dup-ed copy of the set I guess.

Should've probably realized this sooner but the fact that the ids were the same was throwing me off -- I guess I figured a forked process would get its own virtual addresses, so the odds of it being the same one would be low. Didn't think to assume it would just re-use the same virtual address.

tl;dr: id(x) == id(y) doesn't mean x and y are the same object if you're messing with multiprocessing.

来源：https://stackoverflow.com/questions/22392344/set-is-being-modified-and-then-seems-to-magically-revert