Kafka: Cant Create Multiple Stream Consumers

Question

I just got up and running with Kafka 0.8 beta 1. I have a really simple example up and running, the problem is, I can only get one message consumer to work, not several. That

Answer 1

Maybe too late for questioner but could be useful to other developers. Seems you have used only one Partition for couple of Consumers – that's wrong. Quote from Documentation:

Since there are many partitions this still balances the load over many consumer instances. Note however that there cannot be more consumer instances than partitions.

So when you think about Consumers you should think how to divide messages by partitions. In most cases you should use some high-level grouping to it or even let it be random by default.