java - How to use Spring Batch read CSV,process it and write it as a CSV with one row can produce more than one row?

Question

I'm using Spring Batch in order to read a CSV file, process it and write it back after some processing, it is pretty simple to do when there is a one to one relation between the source and target, but according to my business logic, in some cases a row in the input can produce more than one rows in the output files.

This is how the the processor looks like but i couldn't find any information on how to write a Writer for it.

public class CsvRowsProcessor implements ItemProcessor<RowInput, List<RowOutput>>{

    @Override
    public List<RowOutput> process(final RowInput rowInput)  {

        final String id = rowInput.getId();
        final String title = rowInput.getTitle();
        final String description = rowInput.getDescription();
        final RowOutput transformedRowInput = new RowOutput(id, title, description);

        List<RowOutput> rows=new LinkedList<>();
        rows.add(transformedRowInput);
        return rows;
    }

}

Any ideas on how to approach this ?

Current Writer:

@Bean
ItemWriter<RowOutput> csvRowsWriter() {
    FlatFileItemWriter<RowOutput> csvFileWriter = new FlatFileItemWriter<>();
    csvFileWriter.setResource(new FileSystemResource("C:\Users\orenl\IdeaProjects\Spring-Batch-CSV-Example\src\main\resources\outputFile.csv"));
    LineAggregator<RowOutput> lineAggregator = createLineAggregator();
    csvFileWriter.setLineAggregator(lineAggregator);
    csvFileWriter.setHeaderCallback(new FlatFileHeaderCallback() {

        public void writeHeader(Writer writer) throws IOException {
            writer.write("Id,Title,Description");
        }
    });
    return csvFileWriter;
}



private LineAggregator<RowOutput> createLineAggregator() {
    DelimitedLineAggregator<RowOutput> lineAggregator = new DelimitedLineAggregator<>();
    lineAggregator.setDelimiter(",");

    FieldExtractor<RowOutput> fieldExtractor = createFieldExtractor();
    lineAggregator.setFieldExtractor(fieldExtractor);

    return lineAggregator;
}

private FieldExtractor<RowOutput> createFieldExtractor() {
    BeanWrapperFieldExtractor<RowOutput> extractor = new BeanWrapperFieldExtractor<>();
    extractor.setNames(new String[] { "Id", "Title", "Description" });
    return extractor;
}

@Bean
public Step csvFileToFileStep() {
    return stepBuilderFactory.get("csvFileToFileStep")
            .<RowInput ,RowOutput>chunk(1)
            .reader(csvRowsReader())
            .processor(csvRowsProcessor())
            .writer(csvRowsWriter())
            .build();
}

@Bean
Job csvFileToCsvJob(JobCompletionNotificationListener listener) {
    return jobBuilderFactory.get("csvFileToCsvJob")
            .incrementer(new RunIdIncrementer())
            .listener(listener)
            .flow(csvFileToFileStep())
            .end()
            .build();
}

Looking for a solution which will allow me to use setLineAggregator(lineAggregator); but will not write all the rows as one line.

Answer 1

You need to define a writer that accepts a list of lists:

class MyItemWriter implements ItemWriter<List<RowOutput>> {
    public void write(List<? extends List<RowOutput>> items) {
         // write items here
    }    
}

A similar question can be found here: Using an ItemWriter with List of Lists.

Hope this helps.