Kafka Streams - How to better control partitioning of internally created state store topic?

前端 未结 1 1892
野性不改
野性不改 2021-01-25 11:42

State stores in Kafka Streams are created internally. State stores are partitioned by key, but do not allow to provide partitioning other than by key (to my knowledge)

相关标签:
1条回答
  • 2021-01-25 12:20

    The number of changelog topic partitions depends in the number of input topic partitions and you cannot change it, because the state is shared based on this number (ie, there is one partitions by shard).

    If you want to partition changelog topic by some attribute, you must set it as key. In your case, you must set SpecificKey as message key. It's not allowed to change the partitioning because it would "break" Kafka Streams leading to incorrect result.

    0 讨论(0)
提交回复
热议问题