apache-zookeeper

问题 This is a followup question to "Where do zookeeper store Kafka cluster and related information?" based on the answer provided by Armando Ballaci. Now it's clear that consumer offsets are stored in the Kafka cluster in a special topic called __consumer_offsets . That's fine, I am just wondering how does the retrieval of these offsets work. Topics are not like RDBS over which we can query for arbitrary data based on a certain predicate. Ex - if the data is stored in an RDBMS, probably a query

问题 By saying cluster info, I am referring to information like subscribed consumers/consumer groups read and committed offsets leaders and followers of a partition topics on the server etc. Does zookeeper keep this info in its own db (though I never heard of zookeeper having any db of its own till date) or it stores this info in the Kafka cluster on some topics etc? EDIT: and a follow up Q: How does Zookeeper retrive the consumer offsets from __consumer_offsets topic? 回答1: The ZooKeeper Data

问题 My understanding is that Zookeeper is often used to solve the problem of "keeping track of which node plays a particular role" in a distributed system (e.g. master node in a DB or in a MapReduce cluster, etc). For simplicity, say we have a DB with one master and multiple replicas and the current master node in the DB goes down. In this scenario, one would, in principle, make one of the replica nodes a new master node. At this point my understanding is: If we didn't have Zookeeper The

问题 I read how current master election algorithms like Raft, Paxos or Zab elect master on a cluster and couldn't understand why they use sophisticated algorithms instead of simple bully algorithm. I'm developing a cluster library and use UDP Multicast for heartbeat messages. Each node joins a multicast address and also send datagram packets periodically to that address. If the nodes find out there is a new node that sends packets to this multicast address, the node is simply added to cluster and

问题 I would like to use the embedded Zookeeper 3.4.9 that come with Kafka 10.2, and not install Zookeeper separately. Each Kafka broker will always have a 1:1 Zookeeper on localhost. So if I have 5 brokers on hosts A, b, C, D and E, each with a single Kafka and Zookeeper instance running on them, then is it sufficient to just run the Zookeeper provided with Kafka? What downsides or configuration limitations if any does the embedded 3.4.9 Zookeper have compared to the standalone version? 回答1:

问题 While executing any command in hbase shell, I am receiving the following error "ERROR: KeeperErrorCode = NoNode for /hbase/master" in hbase shell. Started HBASE : HOSTCHND:hbase-2.0.0 gvm$ ./bin/start-hbase.sh localhost: running zookeeper, logging to /usr/local/Cellar/hbase-2.0.0/bin/../logs/hbase-gvm-zookeeper-HOSTCHND.local.out running master, logging to /usr/local/Cellar/hbase-2.0.0/logs/hbase-gvm-master-HOSTCHND.local.out : running regionserver, logging to /usr/local/Cellar/hbase-2.0.0

问题 I am able to set up zookeeper cluster on 1 machine with 3 different ports, but when I do the same with different IP to have zookeeper instance on different machines, it throws following error: 2014-11-20 12:16:24,819 [myid:1] - INFO [main:QuorumPeerMain@127] - Starting quorum peer 2014-11-20 12:16:24,827 [myid:1] - INFO [main:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:2181 2014-11-20 12:16:24,842 [myid:1] - INFO [main:QuorumPeer@959] - tickTime set to 2000 2014-11-20 12:16:24

问题 I am trying to just run a TestingServer to be able to write unit tests for it. val zkServer = new TestingServer(2182) println(zkServer.getPort) My pom has the following dependencies related to Curator, I have tried version 4.2.0 and 4.3.0, both throw the same error: <dependency> <groupId>org.apache.curator</groupId> <artifactId>curator-framework</artifactId> <version>${apache.curator.version}</version> </dependency> <!-- https://mvnrepository.com/artifact/org.apache.curator/curator-recipes --

问题 Installed zookeeper 3.5.6 bin on windows. Getting error: Unable to start AdminServer, exiting abnormally org.apache.zookeeper.server.admin.AdminServer$AdminServerException: Problem starting AdminServer on address 0.0.0.0, port 8080 and command URL /commands Where do I find the AdminServer configuration properties to fix this. In my conf directory the .cfg does not have anything, and I cant find anything anywhere else either?? The documentation is referring to zookeeper.admin.serverPort

问题 As of now I am creating a topic one by one by using below command. sh ./bin/kafka-topics --create --zookeeper localhost:2181 --topic sdelivery --replication-factor 1 --partitions 1 Since I have 200+ topics to be created. Is there any way to create a list of topic with a single command? I am using 0.10.2 version of Apache Kafka. 回答1: This seems like more of a unix/bash question than a Kafka one: the xargs utility is specifically designed to run a command repeatedly from a list of arguments. In