[CAMEL-7080] Add support of lazy load to csv data format

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.12.3, 2.13.0
• Patch Info: Patch Available
• Estimated Complexity: Unknown
• Assignee: Willem Jiang
• Reporter: Daneel S. Yaitskov
• Created: 18/Dec/13 19:28
• Updated: 08/Jun/14 12:59
• Resolved: 22/Dec/13 04:06

Description

CsvDataFormat uses greedy loading from a stream. This is not applicable for huge files i.e. all rows cannot fit into available memory.

I added an option to create a row iterator instead list.
Csv format is not line oriented so split("\n") workaround doesn't work always. This patch adds ability to Camel to process infinite csv files with multiline rows.

Pull request:
https://github.com/yaitskov/camel/commits/csv-iterator

P.S.
1) I found that Bindy does similar work related with parsing csv file into list of why. And I don't see any clue why it doesn't delegate this job to csv file format. Roughly speaking there is code duplication.

2) I didn't find camel-csv component on the ticket form.

Issue Links

• relates to: CAMEL-7099 csv data format - Make use of buffered I/O while unmarshalling
• relates to: CAMEL-7098 csv data format - Simplify and clean-up the unmarshalling logic