If you have less consumers than partitions, what happens?

Question

If you have less consumers than partitions, does that simply mean you will not consume all the messages on a given topic?

In a cloud environment, how are you suppose to

Answer 1

1) No that means you will one consumer handling more than one consumer. 2) Kafka never assigns same partition to more than one consumer because that will violate order guarantee within a partition. 3) You could implement ConsumerRebalanceListener, in your client code that gets called whenever partitions are assigned or revoked from consumer.

You might want to take a look at this article specically "Assigning partitions to consumers" part. In that i have a sample where you create topic with 3 partitions and then a consumer with ConsumerRebalanceListener telling you which consumer is handling which partition. Now you could play around with it by starting 1 or more consumers and see what happens. The sample code is in github

http://www.javaworld.com/article/3066873/big-data/big-data-messaging-with-kafka-part-2.html