Spring Batch - Reading multiple line log message

Question

I am facing a problem to read multi-line log message as a single message in our spring batch application configured with spring integration, this application has to read mul

Answer 1

Your problem does not feet in to the RecordSeparatorPolicy.isEndOfRecord(String) paradigm. isEndOfRecored works nicely when the lined ending is placed in the last line.
For example in DefaultRecordSeparatorPolicy it makes sure that you have an even count of quotes. The last quote includes in the required record. In your case you will be over-reading one line.

Your basic idea of using postProcess and preProcess might work, but you still get FlatFileParseException from FlatFileItemReader on the last line when you reach the EOL and readline returns null see applyRecordSeparatorPolicy(String line) in FlatFileItemReader.

  private String applyRecordSeparatorPolicy(String line) throws IOException {

        String record = line;
        while (line != null && !recordSeparatorPolicy.isEndOfRecord(record)) {
            line = this.reader.readLine();
            if (line == null) {

                if (StringUtils.hasText(record)) {
                    // A record was partially complete since it hasn't ended but
                    // the line is null
                    throw new FlatFileParseException("Unexpected end of file before record complete", record, lineCount);
                }
                else {
                    // Record has no text but it might still be post processed
                    // to something (skipping preProcess since that was already
                    // done)
                    break;
                }
            }
            else {
                lineCount++;
            }
            record = recordSeparatorPolicy.preProcess(record) + line;
        }

        return recordSeparatorPolicy.postProcess(record);

    }

In such case your output file will be missing lines based on the commit-interval and isEndOfRecord logic.

So basically I suggest using a different approach, did bellabax solution worked for you?

Answer 2

Write your own ItemReader as described in multiorder-line example or as described in this post.