How to create topics in apache kafka?

Question

What is the bestway to create topics in kafka?

• How many replicas/partitions to be defined when we create topics?

In the new producer API, when i try

Answer 1

Partition number determines the parallelism of the topic since one partition can only be consumed by one consumer in a consumer group. For example, if you only have 10 partitions for a topic and 20 consumers in a consumer group, 10 consumers are idle, not receiving any messages. The number really depends on your application, but 1-1000s are all reasonable.

Replica number is determined by your durability requirement. For a topic with replication factor N, Kafka can tolerate up to N-1 server failures without losing any messages committed to the log. 3 replicas are common configuration. Of course, the replica number has to be smaller or equals to your broker number.

auto.create.topics.enable property controls when Kafka enables auto creation of topic on the server. If this is set to true, when applications attempt to produce, consume, or fetch metadata for a non-existent topic, Kafka will automatically create the topic with the default replication factor and number of partitions. I would recommend turning it off in production and creating topics in advance.