how to read both comma separated and pipe line separated csv file in a single item reader in spring batch

问题

I am new to sprig batch. I have a folder which contain multiple csv file, I have implemented MultiResourceItemReader () to read those file . It is working only if all csv file are pipe line ("|") separated.

I want to read both comma (",") separated csv and pipe line separated csv using single reader. Is it possible ? if yes how ?

Here is my code

@Bean
@StepScope
public MultiResourceItemReader<Person> multiResourceItemReader(@Value("#{jobParameters[x]}") String x,@Value("#{jobParameters[y]}") String y,@Value("#{jobParameters[z]}") String z) {

    Resource[] resourcessss = null;
    ClassLoader cl = this.getClass().getClassLoader();
    ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver(cl);
    try {
        resourcessss = resolver.getResources("file:" + z);
    }catch(Exception e) {

    }
    MultiResourceItemReader<Person> resourceItemReader = new MultiResourceItemReader<Person>();
    resourceItemReader.setResources(resourcessss);
    resourceItemReader.setDelegate(reader());

    return resourceItemReader;
}



@Bean
public FlatFileItemReader<Person> reader() {
    FlatFileItemReader<Person> reader = new FlatFileItemReader<Person>();
    reader.setLineMapper(new DefaultLineMapper() {
        {
            setLineTokenizer(new DelimitedLineTokenizer() {
                {
                    setNames(new String[]{"Id","postCode"});
                }
                {
                    setDelimiter("|");
                }
            });
            setFieldSetMapper(new BeanWrapperFieldSetMapper<Person>() {
                {
                    setTargetType(Person.class);
                }
            });
        }
    });
    return reader;
}

回答1:

Take a look at the PatternMatchingCompositeLineTokenizer. There, you can use a Pattern to identify what records get parsed by what LineTokenizer. In your case, you'd have one Pattern that identifies comma delimited records and map them to the tokenizer that parses via commas. You'd also have a Pattern that identifies records delimited by pipes and maps those to the appropriate LineTokenizer. It would look something like this:

    @Bean
    public LineTokenizer compositeLineTokenizer() throws Exception {
        DelimitedLineTokenizer commaTokenizer = new DelimitedLineTokenizer();

        commaTokenizer.setNames("a", "b", "c");
        commaTokenizer.setDelimiter(",");
        commaTokenizer.afterPropertiesSet();

        DelimitedLineTokenizer pipeTokenizer = new DelimitedLineTokenizer();

        pipeTokenizer.setNames("a", "b", "c");
        pipeTokenizer.setDelimiter("|");

        pipeTokenizer.afterPropertiesSet();

        // I have not tested the patterns here so they may need to be adjusted
        Map<String, LineTokenizer> tokenizers = new HashMap<>(2);
        tokenizers.put("*,*", commaTokenizer);
        tokenizers.put("*|*", pipeTokenizer);

        PatternMatchingCompositeLineTokenizer lineTokenizer = new PatternMatchingCompositeLineTokenizer();

        lineTokenizer.setTokenizers(tokenizers);

        return lineTokenizer;
    }

来源：https://stackoverflow.com/questions/62413263/how-to-read-both-comma-separated-and-pipe-line-separated-csv-file-in-a-single-it

标签

java

Spring

spring-batch

batch-processing