问题
I was wondering how I could determine in my ItemWriter
, whether Spring Batch was currently in chunk-processing-mode or in the fallback single-item-processing-mode. In the first place I didn't find the information how this fallback mechanism is implemented anyway.
Even if I haven't found the solution to my actual problem yet, I'd like to share my knowledge about the fallback mechanism with you.
Feel free to add answers with additional information if I missed anything ;-)
回答1:
The implementation of the skip mechanism can be found in the FaultTolerantChunkProcessor and in the RetryTemplate.
Let's assume you configured skippable exceptions but no retryable exceptions. And there is a failing item in your current chunk causing an exception.
Now, first of all the whole chunk shall be written. In the processor's write()
method you can see, that a RetryTemplate
is called. It also gets two references to a RetryCallback
and a RecoveryCallback
.
Switch over to the RetryTemplate
. Find the following method:
protected <T> T doExecute(RetryCallback<T> retryCallback, RecoveryCallback<T> recoveryCallback, RetryState state)
There you can see that the RetryTemplate
is retried as long as it's not exhausted (i.e. exactly once in our configuration). Such a retry will be caused by a retryable exception. Non-retryable exceptions will immediately abort the retry mechanism here.
After the retries are exhausted or aborted, the RecoveryCallback
will be called:
e = handleRetryExhausted(recoveryCallback, context, state);
That's where the single-item-processing mode will kick-in now!
The RecoveryCallback (which was defined in the processor's write()
method!) will put a lock on the input chunk (inputs.setBusy(true)
) and run its scan()
method. There you can see, that a single item is taken from the chunk:
List<O> items = Collections.singletonList(outputIterator.next());
If this single item can be processed by the ItemWriter
correctly, than the chunk will be finished and the ChunkOrientedTasklet
will run another chunk (for the next single items). This will cause a regular call to the RetryCallback
, but since the chunk has been locked by the RecoveryTemplate
, the scan()
method will be called immediately:
if (!inputs.isBusy()) {
// ...
}
else {
scan(contribution, inputs, outputs, chunkMonitor);
}
So another single item will be processed and this is repeated, until the original chunk has been processed item-by-item:
if (outputs.isEmpty()) {
inputs.setBusy(false);
That's it. I hope you found this helpful. And I even more hope that you could find this easily via a search engine and didn't waste too much time, finding this out by yourself. ;-)
回答2:
A possible approach to my original problem (the ItemWriter would like to know, whether it's in chunk or single-item mode) could be one of the following alternatives:
- Only when the passed chunk is of size one, any further checks have to be done
When the passed chunk is a
java.util.Collections.SingletonList
, we would be quite sure, since theFaultTolerantChunkProcessor
does the following:List items = Collections.singletonList(outputIterator.next());
Unfortunately, this class is private and so we can't check it with
instanceOf
.In reverse, if the chunk is an
ArrayList
we could also be quite sure, since the Spring Batch'sChunk
class uses it:private List items = new ArrayList();
- One blurring left would be buffered items read from the execution context. But I'd expect those to be ArrayLists also.
Anyway, I still find this method too vague. I'd rather like to have this information provided by the framework.
An alternative would be to hook my ItemWriter
in the framework execution. Maybe ItemWriteListener.onWriteError()
is appropriate.
Update: The onWriteError()
method will not be called if you're in single-item mode and throw an exception in the ItemWriter
. I think that's a bug a filed it: https://jira.springsource.org/browse/BATCH-2027
So this alternative drops out.
Here's a snippet to do the same without any framework means directly in the writer
private int writeErrorCount = 0;
@Override
public void write(final List<? extends Long> items) throws Exception {
try {
writeWhatever(items);
} catch (final Exception e) {
if (this.writeErrorCount == 0) {
this.writeErrorCount = items.size();
} else {
this.writeErrorCount--;
}
throw e;
}
this.writeErrorCount--;
}
public boolean isWriterInSingleItemMode() {
return writeErrorCount != 0;
}
Attention: One should rather check for the skippable exceptions here and not for Exception
in general.
来源:https://stackoverflow.com/questions/16567432/how-is-the-skipping-implemented-in-spring-batch