zeromq | 易学教程

分布式进阶(二十) Kafka简介

阅读更多关于分布式进阶(二十) Kafka简介

一、简介 1.1　概述 Kafka是最初由Linkedin公司开发，是一个分布式、分区的、多副本的、多订阅者，基于zookeeper协调的分布式日志系统（也可以当做MQ系统），常见可以用于web/nginx日志、访问日志，消息服务等等，Linkedin于2010年贡献给了Apache基金会并成为顶级开源项目。主要应用场景是：日志收集系统和消息系统。 Kafka主要设计目标如下：以时间复杂度为O(1)的方式提供消息持久化能力，即使对TB级以上数据也能保证常数时间的访问性能。高吞吐率。即使在非常廉价的商用机器上也能做到单机支持每秒100K条消息的传输。支持Kafka Server间的消息分区，及分布式消费，同时保证每个partition内的消息顺序传输。同时支持离线数据处理和实时数据处理。 Scale out:支持在线水平扩展 1.2　消息系统介绍一个消息系统负责将数据从一个应用传递到另外一个应用，应用只需关注于数据，无需关注数据在两个或多个应用间是如何传递的。分布式消息传递基于可靠的消息队列，在客户端应用和消息系统之间异步传递消息。有两种主要的消息传递模式：点对点传递模式、发布-订阅模式。大部分的消息系统选用发布-订阅模式。 Kafka就是一种发布-订阅模式。 1.3　点对点消息传递模式在点对点消息系统中，消息持久化到一个队列中。此时

ZMQ socket connect timeout

阅读更多关于 ZMQ socket connect timeout

问题 I'm trying to connect a socket to an endpoint until the socket receives data from that endpoint. This is because the endpoint might not exist at that time. Currently the connect stalls, i'm guessing because it can't resolve the hostname and that takes a while. Is there any way to set a timeout on a socket connect? import zmq import time endpoint = 'tcp://doesnt_exist:12345' ctx = zmq.Context.instance() s = ctx.socket(zmq.SUB) t = time.time() try: s.connect(endpoint) except Exception: pass

zeromq with protobuf segmentation fault while parsing in c++

阅读更多关于 zeromq with protobuf segmentation fault while parsing in c++

问题 I am using zeromq with protobuf to send/recieve messages but code was crashing on receiver end with Segmentation fault (core dumped) error while parsing the received data. Scan is my message. sender.cpp Scan proto_ls_msg; proto_ls_msg.set_angle_min(0.0); proto_ls_msg.set_angle_max(180.5); std::string ls_msg_str; proto_ls_msg.SerializeToString(&ls_msg_str); zmq::message_t request (ls_msg_str.size()); memcpy (request.data(), ls_msg_str.c_str(),ls_msg_str.size()); socket.send (request);

npm install zmq issue

阅读更多关于 npm install zmq issue

问题 I am doing npm install zmq but stuck in strange error, please see below C:\Users\Administrator>npm install zmq MSBUILD : error MSB4132: The tools version "2.0" is unrecognized. Available too ls versions are "4.0". gyp ERR! stack Error: C:\Windows\Microsoft.NET\Framework\v4.0.30319\msbuild.exe failed with exit code: 1 gyp ERR! stack at ChildProcess.onExit (C:\Program Files\nodejs\node_modules\ npm\node_modules\node-gyp\lib\build.js:267:23) gyp ERR! stack at ChildProcess.EventEmitter.emit

Django and SaaS + Ztask for background tasks

阅读更多关于 Django and SaaS + Ztask for background tasks

问题 I asked a question before on how to create a SaaS application using Django Django and SaaS. How to use separate database for each Django site? Now when a user creates a new instance for himself I simply create a new settings.py file for him, and create an apache configuration for his subdomain. Now when a user tries to access his instance he can only see his own database, so we have total separation of data. Now there is a problem when trying to do background processes while having these

How to receive data continuously and update HTML table?

阅读更多关于 How to receive data continuously and update HTML table?

问题 I am new to Web Development. I'm creating a web application where I need to use ZMQ Library for receiving data continuously and update a html table in real-time. I tried a lot and just succeeded in receiving a single data packet. Later I'm calling the function recursively to get all data packets. But, by the time I call the function, I already lost some packets in the gap. Following is the javascript function I'm using in index.php and test.php for receiving data using ZMQ: Javascript

MQL5 Zmq PUB socket not being received by Python zmq SUB socket

阅读更多关于 MQL5 Zmq PUB socket not being received by Python zmq SUB socket

问题 I'm trying to set up a PUB socket in MQL5 and a SUB socket in Python that will receive messages. I have this in MQL5: #include <Zmq/Zmq.mqh> Context context("helloworld"); Socket socket(context,ZMQ_PUB); string BROKER; int OnInit() { if (socket.bind("tcp://*:5556")) { Print("Error"); } else Print("Bound"); BROKER = AccountInfoString(ACCOUNT_COMPANY); return(INIT_SUCCEEDED); } void OnTick() { MqlTick last_tick; string str; if(SymbolInfoTick(Symbol(),last_tick)) { StringConcatenate(str, BROKER,

return value from zmq socket

阅读更多关于 return value from zmq socket

问题 I am using Socket.IO to have a client webpage fetch data dynamically from my Node.JS server. On the server-side, I am forwarding the request to a Python backend using the zeromq socket library (REQ/REP structure, see the rrclient.js example from zguide). However, since my server is acting as the socket.io server, I have to wrap my response as follows in order to send it back to the specific client: socket.on('app_request',function(appRequest){ //route this request to the backend. var results

Linker problem with ZeroMQ and Visual C++ 2008

阅读更多关于 Linker problem with ZeroMQ and Visual C++ 2008

问题 I'm having a problem trying to use ZeroMQ in my new application. Please note: I have no problem downloading and building ZeroMQ itself. I downloaded the ZeroMQ ZIP file, opened the project/solution file in Visual Studio/C++ 2008 and built the ZeroMQ library just fine. The /lib directory of the ZeroMQ install folder contains the .lib, .dll, and other files, so all is well there as far as I know. My problem is that I am trying to build a simple project consisting of the HelloWorld server

Reason for zmq error code 156384763

阅读更多关于 Reason for zmq error code 156384763

问题 I am using zmq req/rep pattern communication. The implementation is pretty simple, the req sends some data and waits on recv. The rep receives the data, process and reply back. //REQ zmq_connect zmq_send zmq_recv //blocking zmq_close //REP zmq_bind while(true) { while(data_received) { //miscellaneous process zmq_recv //non-blocking Print zmq_error_no if zmq_recv fails } zmq_send } In the REP side, during zmq_recv timeout zmq_error_no 11 will be printed. But sometimes i am getting error no