Python zeromq — Multiple Publishers To a Single Subscriber?

Question

I\'d like to write a python script (call it parent) that does the following:

(1) defines a multi-dimensional numpy

Answer 1

In ZeroMQ there can only be one publisher per port. The only (ugly) workaround is to start each child PUB socket on a different port and have the parent listen on all those ports.

but the pipeline pattern describe on 0MQ, user guide is a much better way to do this.

Answer 2

The sub channel doesn't have to be the one to bind, so you can have the subscriber bind, and each of the children pub channels can connect to that and send their messages. In this particular case, I think the multiprocessing module is a better fit, but I thought it bore mentioning:

import zmq
import threading

# So that you can copy-and-paste this into an interactive session, I'm
# using threading, but obviously that's not what you'd use

# I'm the subscriber that multiple clients are writing to
def parent():
    context = zmq.Context()
    socket = context.socket(zmq.SUB)
    socket.setsockopt(zmq.SUBSCRIBE, 'Child:')
    # Even though I'm the subscriber, I'm allowed to get this party 
    # started with `bind`
    socket.bind('tcp://127.0.0.1:5000')

    # I expect 50 messages
    for i in range(50):
        print 'Parent received: %s' % socket.recv()

# I'm a child publisher
def child(number):
    context = zmq.Context()
    socket = context.socket(zmq.PUB)
    # And even though I'm the publisher, I can do the connecting rather
    # than the binding
    socket.connect('tcp://127.0.0.1:5000')

    for data in range(5):
        socket.send('Child: %i %i' % (number, data))
    socket.close()

threads = [threading.Thread(target=parent)] + [threading.Thread(target=child, args=(i,)) for i in range(10)]
for thread in threads:
    thread.start()

for thread in threads:
    thread.join()

In particular, the Core Messaging Patterns part of the documentation discusses the fact that for the patterns, either side can bind (and the other connect).

Answer 3

I think it makes more sense to use PUSH/PULL sockets, as you have a standard Ventilator - Workers - Sink scenario, except that the Ventilator and the Sink are the same process.

Also, consider using the multiprocessing module instead of ZeroMQ. It will probably be a bit easier.