Unexpected in Spring partition when using synchronized

前端 未结 1 2013
遥遥无期
遥遥无期 2021-01-03 16:14

I am using Spring Batch and Partition to do parallel processing. Hibernate and Spring Data Jpa for db. For the partition step, the reader, processor and writer have stepscop

相关标签:
1条回答
  • 2021-01-03 16:22

    Since you made the item processor step-scoped, you don't really need synchronization as each step will have its own instance of the processor.

    But it looks like you have a design problem rather than an implementation issue. You are trying to sychronize threads to act in a certain order in a parallel setup. When you decide to go parallel and divide the data into partitions and give each worker (either local or remote) a partition to work on, you must admit that these partitions will be processed in an undefined order and that there should be no relation between records of each partition or between the work done by each worker.

    When 1 thread execute that method, it will create 1 account and save it. If next thread comes in and try to save the same account, it should find the account created by first thread and do one update. But now it doesn't happen, instead findIfExist return null and it try to do another insert of duplicate data

    That's because the transaction of thread1 may not be committed yet, hence thread2 won't find the record you think have been inserted by thread1.

    It looks like you are trying to create or update some accounts with a partitioned setup. I'm not sure if this setup is suitable for the problem at hand.

    As a side note, I would not call accountRepo.save(account); in an item processor but rather do that in an item writer.

    Hope this helps.

    0 讨论(0)
提交回复
热议问题