pyzmq

Which ZeroMQ pattern is best of an asynchronous pair of sockets?

↘锁芯ラ 提交于 2019-12-10 23:29:16
问题 I have a server (running on Amazon) and a single client that connects to it. After a connect has been established the client and server exclusively communicate with each other and send messages. e.g. 1. Client -> Server 2. Client -> Server 3. Client <- Server 4. Client -> Server 5. Client <- Server 6. Client <- Server The client might lost connectivity and re-connect after some time and resume message sending. Also what are the implications of the order of messages? Could #2 arrive before #1?

ZeroMQ: HWM on PUSH does not work

对着背影说爱祢 提交于 2019-12-10 19:03:25
问题 I am trying to write a server/client script with a server that vents the tasks, and multiple workers that execute it. The problem is that my ventilator has so many tasks that it would fill up the memory in a heartbeat. I tried to set the HWM before it binds, but with no success. It just keeps on sending messages as soon as a worker connects, completely disregarding the HWM that was set. I also have a sink that keeps record of the tasks that were done. server.py import zmq def ventilate():

Accessing celery worker instance inside the task

大城市里の小女人 提交于 2019-12-10 18:26:49
问题 I want to use jupyter kernels in side the celery worker. There will be one Jupyter Kernel for each Celery Worker. To achieve it I am overriding the default Worker class of the celery, at the initialisation of the worker I am starting the jupyter kernel and with the stop method I am shutting down the jupyter kernel. The current problem I am facing is how can I access that kernel instance inside the task while the task is running ? Is there any better way to override the Worker class definition

pyzmq recv_json can't decode message sent by send_json

荒凉一梦 提交于 2019-12-10 17:38:22
问题 Here is my code with the extraneous stuff stripped out: coordinator.py context = zmq.Context() socket = context.socket(zmq.ROUTER) port = socket.bind_to_random_port(ZMQ_ADDRESS) poller = zmq.Poller() poller.register(socket, zmq.POLLIN) while True: event = poller.poll(1) if not event: continue process_id, val = socket.recv_json() worker.py context = zmq.Context() socket = context.socket(zmq.DEALER) socket.connect('%s:%s' % (ZMQ_ADDRESS, kwargs['zmq_port'])) socket.send_json( (os.getpid(), True

Identifying the origin of ZMQ messages?

三世轮回 提交于 2019-12-10 17:29:24
问题 If I receive a message via the recv() method on a ZeroMQ (0MQ) socket... data = s.recv() ...is there any way for me to get at the value of getpeername() for the underlying socket? My goal is to identify the origin of the message in a way that does not rely on the sender to provide accurate information. I'm using ZMQ (via Python) to collect host metrics, and the address of the sender from the perspective of the receiver is a useful identifier. Or is this just a Bad Idea? 回答1: No, you cannot

Zeromq which socket should bind on PubSub pattern

浪尽此生 提交于 2019-12-10 13:19:57
问题 I have been reading about ZeroMQ more specifically about NetMQ and almost every Pub/Sub examples I saw used to Bind the Publisher socket and then the Subscriber socket connects to the other. So i'm wondering if it is possible to do the reverse, i mean Bind the Subscriber socket and then publishers connect to it. Is this possible ? (I didn't found anything clear on documentation) What are the disadvantages using this connection strategy ? Any help will be usefull. 回答1: Yes, you can reverse it

How to check whether python package is installed or not in Docker?

三世轮回 提交于 2019-12-08 21:43:44
问题 I used Dockerfile successfully built a container. However, my code doesn't work in the container. It does work if I install all the packages manually. I'm assuming I messed up something that cause docker didn't install the packages properly. So, I want to check whether python package is installed or not in Docker container. What is the best way to check it? The Dockerfile I used: # Update the sources list RUN sudo apt-get update # Install basic applications RUN sudo apt-get install -y tar git

ZeroMQ Subscribers not receiving message from Publisher over an inproc: transport class

偶尔善良 提交于 2019-12-08 09:19:46
问题 I am fairly new to pyzmq . I am trying to understand inproc: transport class and have created this sample example to play with. It looks a Publisher instance is publishing messages but Subscriber instances are not receiving any. In case I move Subscriber instances into a separate process and change inproc: to a tcp: transport class, the example works. Here is the code: import threading import time import zmq context = zmq.Context.instance() address = 'inproc://test' class Publisher(threading

ZeroMQ N to N async pattern in Python

 ̄綄美尐妖づ 提交于 2019-12-08 08:01:30
问题 N-proxy-N Pub-Sub Similar to the question N to N async pattern in ZeroMQ?, but which unfortunately never received an answer with working code. I'm trying to implement Pub-Sub network as described in the guide: http://zguide.zeromq.org/py:all#The-Dynamic-Discovery-Problem (a small message broker in the style of N-proxy-N). Unfortunately, the guide doesn't provide any code examples. I've tried to implement an Hello World example using PyZMQ, I think I'm close, but I'm facing some errors I don't

0mq: pubsub latency continually growing with messages?

早过忘川 提交于 2019-12-08 06:09:01
问题 pub.py import zmq import random import sys import time port = "5556" if len(sys.argv) > 1: port = sys.argv[1] int(port) context = zmq.Context() socket = context.socket(zmq.PUB) socket.bind("tcp://*:%s" % port) topic = 10001 while True: msgdata = time.time() socket.send("%d %d" % (topic, msgdata)) print "topic:%d, msg:%.5f" % (topic, msgdata) time.sleep(1) sub.py import sys import zmq import time port = "5556" if len(sys.argv) > 1: port = sys.argv[1] int(port) if len(sys.argv) > 2: port1 = sys