How to ensure all step partitions are finished before going into the next job step

问题

In my job I have a database export to csv which is partitioned. The next step merges those files into a single file using MultiResourceItemReader.

My problem is that the files don't seem to be available when the merge step starts. I get this exception

java.lang.IllegalStateException: No resources to read. Set strict=false if this is not an error condition.
    at org.springframework.batch.item.file.MultiResourceItemReader.open(MultiResourceItemReader.java:169)
    at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:96)
    at org.springframework.batch.core.step.item.ChunkMonitor.open(ChunkMonitor.java:114)
    at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:96)
    at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:310)
    at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:195)
    at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148)
    at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:64) 

but then after the job fails all the part csv files are available in the export directory. In the spring batch reference it actually lists this very architectural consideration on page 16:

Also the architecture should take into consideration the consolidation of partitions. Key questions include: • Must all the partitions be finished before going into the next job step?

I've searched the document and the web for an answer to how to ensure all step partitions are finished before going into the next job step and can't seem to find the answer.

It must be a common consideration though, obviously I'm missing something, big.

回答1:

By default, only once all the step-parts of a partition are complete will the step be marked complete. Are you sure you next step is looking for the files in the correct directory?

You can turn up logging and it should provide details about when the step complete and when the next step starts

来源：https://stackoverflow.com/questions/27333440/how-to-ensure-all-step-partitions-are-finished-before-going-into-the-next-job-st