问题
I don't want to write processed KStream to another topic, I directly want to write enriched KStream to database. How should I proceed?
回答1:
You can implement a custom Processor
that opens a DB connection and apply it via KStream#process()
. Cf. https://docs.confluent.io/current/streams/developer-guide.html#applying-processors-and-transformers-processor-api-integration
Note, you will need to do sync writes into your DB to guard against data loss.
Thus, not writing back to a topic has multiple disadvantages:
- reduced throughput because of sync writes
- you cannot use exactly-once semantics
- coupling your application with the database (if DB goes down, your app goes down, too, as it can't write its results anymore)
Therefore, it's recommended to write the results back into a topic and use Connect API to get the data into your database.
来源:https://stackoverflow.com/questions/46524930/how-to-process-a-kafka-kstream-and-write-to-database-directly-instead-of-sending