how to set group name when consuming messages in kafka using command line

Question

Any idea how to set group name when consuming messages in kafka using command line.

I tried with the following command :

bin/kafka-console-consumer.sh --         


        

Answer 1

The simplest solution is:

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic nil_RF2_P2 --from-beginning --consumer-property group.id=test1

In case you specify the flag --from-beginning just remember that the consumer group should not have consumed any records in the past or else your consumer will start consuming from the earliest not consumed record by the specified group (and not from the actual beginning, as you may incorrectly assume).

Answer 2

if you want change group id without lost offset of record you have get offset manually of current Group.id and set to new run consumer that have new id. if don't have any control to get offset in consumer instance you can run this command.

/bin/kafka-consumer-groups.sh --new-consumer --bootstrap-server <ip_address>:<Broker_port>  --group Group_name --describe

and then you can seek data from specific offset. pay attention you should call seek after call poll، Assign command not work. also you can see my sample of code in github

Example here

Answer 3

You can use the --group option like this (tested with Kafka 2.0.0):

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --group test-consumer --topic test --from-beginning

Answer 4

Got the answer to change the groupname from command prompt!!

steps:

1. create a new consumer.properties file, say consumer1.properties.
2. change group.id=<Give a new group name> in the consumer1.properties.
3. bin/kafka-console-consumer.sh --new-consumer --bootstrap-server localhost:9092 --topic topicname --from-beginning --consumer.config config/consumer1.properties --delete-consumer-offsets

Answer 5

If you are using bash then you can use its process substitution capability.

bin/kafka-console-consumer.sh --zookeeper localhost:2181 \
--topic nil_RF2_P2 --from-beginning \
--consumer.config <(echo group.id=test1)