Apache Kafka Streams Materializing KTables to a topic seems slow

Question

I\'m using kafka stream and I\'m trying to materialize a KTable into a topic.

It works but it seems to be done every 30 secs or so.

How/When does Kafka Stream

Answer 1

This is controlled by commit.interval.ms, which defaults to 30s. More details here: http://docs.confluent.io/current/streams/developer-guide.html

The semantics of caching is that data is flushed to the state store and forwarded to the next downstream processor node whenever the earliest of commit.interval.ms or cache.max.bytes.buffering (cache pressure) hits.

and here:

https://cwiki.apache.org/confluence/display/KAFKA/KIP-63%3A+Unify+store+and+downstream+caching+in+streams