What command shows all of the topics and offsets of partitions in Kafka?

Question

I\'m looking for a Kafka command that shows all of the topics and offsets of partitions. If it\'s dynamically would be perfect. Right now I\'m using java code to see these infor

Answer 1

Kafka ships with some tools you can use to accomplish this.

List topics:

# ./bin/kafka-topics.sh --list --zookeeper localhost:2181
test_topic_1
test_topic_2
...

List partitions and offsets:

# ./bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --broker-info --group test_group --topic test_topic --zookeeper localhost:2181
Group           Topic                  Pid Offset          logSize         Lag             Owner
test_group      test_topic             0   698020          698021          1              test_group-0
test_group      test_topic             1   235699          235699          0               test_group-1
test_group      test_topic             2   117189          117189          0               test_group-2

Update for 0.9 (and higher) consumer APIs

If you're using the new apis, there's a new tool you can use: kafka-consumer-groups.sh.

./bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group count_errors --describe
GROUP                          TOPIC                          PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             OWNER
count_errors                   logs                           2          2908278         2908278         0               consumer-1_/10.8.0.55
count_errors                   logs                           3          2907501         2907501         0               consumer-1_/10.8.0.43
count_errors                   logs                           4          2907541         2907541         0               consumer-1_/10.8.0.177
count_errors                   logs                           1          2907499         2907499         0               consumer-1_/10.8.0.115
count_errors                   logs                           0          2907469         2907469         0               consumer-1_/10.8.0.126