zeromq | 易学教程

通俗地解释一下RPC框架

阅读更多关于通俗地解释一下RPC框架

什么是 RPC ？ RPC (Remote Procedure Call)即远程过程调用，是分布式系统常见的一种通信方法，已经有 40 多年历史。当两个物理分离的子系统需要建立逻辑上的关联时，RPC 是牵线搭桥的常见技术手段之一。除 RPC 之外，常见的多系统数据交互方案还有分布式消息队列、HTTP 请求调用、数据库和分布式缓存等。其中 RPC 和 HTTP 调用是没有经过中间件的，它们是端到端系统的直接数据交互。HTTP 调用其实也可以看成是一种特殊的 RPC，只不过传统意义上的 RPC 是指长连接数据交互，而 HTTP 一般是指即用即走的短链接。 RPC 在我们熟知的各种中间件中都有它的身影。Nginx/Redis/MySQL/Dubbo/Hadoop/Spark/Tensorflow 等重量级开源产品都是在 RPC 技术的基础上构建出来的，我们这里说的 RPC 指的是广义的 RPC，也就是分布式系统的通信技术。RPC 在技术中的地位好比我们身边的空气，它无处不在，但是又有很多人根本不知道它的存在。本地过程调用 RPC就是要像调用本地的函数一样去调远程函数。在研究RPC前，我们先看看本地调用是怎么调的。假设我们要调用函数Multiply来计算lvalue * rvalue的结果: 1 int Multiply(int l, int r) { 2 int y = l * r;

2 minutes for ZMQ pub/sub to connect in kubernetes

阅读更多关于 2 minutes for ZMQ pub/sub to connect in kubernetes

问题 I have a Kubernetes 1.18 cluster using weave as my CNI. I have a ZMQ based pub/sub app and I am often (not always) seeing it take 2 minutes before the subscriber can receive messages from the publisher. This seems to be some sort of socket timeout uniqe to my Kubernetes environment. Here is my trivial ZMQ app example #!/bin/env python2 import zmq, sys, time, argparse, logging, datetime, threading from zmq.utils.monitor import recv_monitor_message FORMAT = '%(asctime)-15s %(message)s' logging

ZeroMQ with NORM - address already in use error was thrown on 2nd .bind() - why?

阅读更多关于 ZeroMQ with NORM - address already in use error was thrown on 2nd .bind() - why?

问题 I'm using ZeroMQ with NACK-Oriented Reliable Multicast ( NORM ) norm:// protocol. The documentation contains only a Python code, so here is my C++ code: PUB Sender : string sendHost = "norm://2,127.0.0.1:5556";// <NormNodeId>,<addr:port> string tag = "MyTag"; string sentMessage = "HelloWorld"; string fullMessage = tag + sentMessage; zmq::context_t *context = new zmq::context_t( 20 ); zmq::socket_t publisher( *context, ZMQ_PUB ); zmq_connect( publisher, sendHost.c_str() ); zmq_send( publisher,

Which bi-directional ZeroMQ pattern should I use for multiple clients connecting to a single server?

阅读更多关于 Which bi-directional ZeroMQ pattern should I use for multiple clients connecting to a single server?

问题 I have multiple ( 1000s ) of clients connecting to a single server and sending some log data. The server analyses the data and responds, if necessary. PUB / SUB is one directional (monitor example ). REQ / REP cannot identify the peer and reply specifically to a peer ( it is more for ACK and such only ). I need to register these clients, identify them and be able to respond after analysis of their logs. With sockets, I have the client sockets, after accept and I can respond using that socket

How to synchronize data amongst devices in Wi-Fi

阅读更多关于 How to synchronize data amongst devices in Wi-Fi

问题 I am developing an app for iOS and Android. The basic functionality is to keep a certain set of data synchronized across all devices in a Wi-Fi network without a central server. Every device can modify that set of data. The current approach is to discover the other devices via Bonjour/Zeroconf and then send the "change messages" to all devices via ZeroMQ. As both Frameworks cause quite a lot of problems to implement I am asking if this is the correct way to accomplish this. I had most of the

C# threading: calling a function on a different thread in its context and returning results

阅读更多关于 C# threading: calling a function on a different thread in its context and returning results

问题 This might be a very naive question: I'm trying to create a client application that uses ZeroMQ for communicating to multiple servers. The client would like to send a large number of requests to these servers and get responses to them (so req-rep pattern). The issue I'm facing is that ZeroMQ sockets should only be used in the threads they are created on. One way is to invoke each of the requests in a new task: inside the task, create a connection, send request and get response. However, the

Updating matplotlib plot in tkinter gui via another process

阅读更多关于 Updating matplotlib plot in tkinter gui via another process

问题 I have an interface that I am writing using tkinter. Inside that interface, I have a matplotlib plot. In the background, I have zmq connecting to a server to get data. I am using python's multiprocessing module with a Queue sharing data and a Process running for all of the zmq communication. In the GUI, I can set up a button to update the plot with the following function: def addData(self): if not self.data.empty(): print "Have Data!" self.hist.addQueue(self.data) self.hist.updatePlot() else:

ZeroMQ, can we use inproc: transport along with pub/sub messaging pattern

阅读更多关于 ZeroMQ, can we use inproc: transport along with pub/sub messaging pattern

问题 Scenario : We were evaluating ZeroMQ (specifically jeroMq ) for an event driven mechanism. The application is distributed where multiple services (both publishers and subscribers are services) can exist either in the same jvm or in distinct nodes, which depends on the deployment architecture. Observation For playing around I created a pub / sub pattern with inproc: as the transport , using jero mq (version :0.3.5) The thread publishing is able to publish (looks like getting published, at

AttributeError: 'str' object has no attribute 'decode'

阅读更多关于 AttributeError: 'str' object has no attribute 'decode'

问题 I am going to install ZeroMQ library in my pc using pip package installer. I use https://learning-0mq-with-pyzmq.readthedocs.org/en/latest/pyzmq/basics.html#installation as my reference. After executing follwing command: pip install pyzmq-static I get this error: C:\Users\MyName>pip install pyzmq-static You are using pip version 7.0.1, however version 7.0.3 is available. You should consider upgrading via the 'pip install --upgrade pip' command. Collecting pyzmq-static Using cached pyzmq

kafka rabbitMQ RockerMq 等消息中间件的对比消息发送新能区别

阅读更多关于 kafka rabbitMQ RockerMq 等消息中间件的对比消息发送新能区别

kafka ：Kafka是LinkedIn开源的分布式发布-订阅消息系统，目前归属于Apache定级项目。Kafka主要特点是基于Pull的模式来处理消息消费，追求高吞吐量，一开始的目的就是用于日志收集和传输。0.8版本开始支持复制，不支持事务，对消息的重复、丢失、错误没有严格要求，适合产生大量数据的互联网服务的数据收集业务。 RabbitMQ:是使用Erlang语言开发的开源消息队列系统，基于AMQP协议实现，主要特征是面向消息队列路由（点对点和发布订阅）可靠性安全性，对数据一致性稳定性高可靠行要求很高，对性能和高吞吐量的要求次之 RocketMQ阿里的开源中间件，是纯Java开发，具有高吞吐量高可用性适合大规模分布式系统的应用特点，思路起源于kafka单并非完全copy kafka，在我看来是对消息的靠靠性以及事务性做了优化测试目的对比Kafka、RabbitMQ、RocketMQ发送小消息(124字节)的性能。这次压测我们只关注服务端的性能指标,所以压测的标准是: 不断增加发送端的压力,直到系统吞吐量不再上升,而响应时间拉长。这时服务端已出现性能瓶颈,可以获得相应的系统最佳吞吐量。测试场景在同步发送场景中，三个消息中间件的表现区分明显： Kafka的吞吐量高达17.3w/s，不愧是高吞吐量消息中间件的行业老大

订阅 zeromq