solr delta import handler timestamp not specific enough

♀尐吖头ヾ 提交于 2019-12-10 11:54:59

问题


I am new to solr and I have a quite basic question about delta-imports. I have several new records by second in my mySQL DB. So when I start an import at second x it is very possible, that I will get some new records in the DB at the very same second after starting the import, but the next time when I start a delta-import it will check the "last_index_time" in dataimport.properties and will import all the records changed after this second x. So I will lose all records which have been changed in second x after starting the last import. And if I am right, it would be same issue even if it is possible to cahange the timestamp from seconds to e.g. milliseconds. The timewindow would be smaller, the amount of lost records would be smaller, but the problem itself would still be there.

I have not found any mention of this issue in the tutorials or anywhere else for that matter. Am I the first one who deals with several records per second, or do i miss something else?

Many thanks in Advance!


回答1:


If it is to handle the exact second, you just check for the records equal to or greater then the last modified time.
Anyways if the the record already exists (identified by Unique Key) it will be overwritten. Solr will update the record by default so no duplicates would be created.



来源:https://stackoverflow.com/questions/18570742/solr-delta-import-handler-timestamp-not-specific-enough

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!