Can we update/Upsert a record in mongodb? data source is kafka

Question

We can update/upsert the record in mongodb BUT is there is any method or function from which we can update or upsert the document directly in mongodb and the source system i

Answer 1

Yes we can update/upsert the data. For update you have to define a parameter in Kafka connector. and whitelist the column on which bases you want to update the record. The property is as followed:

document.id.strategy=com.mongodb.kafka.connect.sink.processor.id.strategy.PartialValueStrategy
value.projection.list=tokenNumber
value.projection.type=whitelist
writemodel.strategy=com.mongodb.kafka.connect.sink.writemodel.strategy.UpdateOneTimestampsStrategy

Answer 2

I was struggling through it finally I got the answer. I used following Mongodb sink connector

And after bugging my head on their document for sometime ,I finally figured out the solution .

This is the exact mongodb sink connector configuration I am using

{
  "name": "mongodbsync",
  "connector.class": "at.grahsl.kafka.connect.mongodb.MongoDbSinkConnector",
  "topics": "alpha-foobar",
  "mongodb.connection.uri": "mongodb://localhost:27017/kafkaconnect?w=1&journal=true",
  "mongodb.document.id.strategy": "at.grahsl.kafka.connect.mongodb.processor.id.strategy.ProvidedInValueStrategy"

}

I left mongodb.writemodel.strategy blank in my configuration so it is taking the default one

I used the use case 2 of following docs from github of the same connector

I was dealing with this scenario ,transferring mysql table data with kafka-jdbc-source connect to mongodb sink.

Also the above strategies can be found in the official docs as well Please free to ask any doubts if you have .Thanks