[CRUNCH-414] The CSV file source needs to be a little more robust when handling multi-line CSV files

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.10.0, 0.8.3
• Fix Version/s: 0.8.4, 0.11.0
• Component/s: Core
• Labels: csv csvparser
• Assignee: mac champion
• Reporter: mac champion
• Created: 05/Jun/14 15:23
• Updated: 19/Sep/14 00:07
• Resolved: 16/Jul/14 01:56

Description

Brandon Inman recently reported an undesriable behavior in the CSV file source group of files. Currently, the CSVLineReader, if reading a malformed CSV file, can enter a state where it is perpetually waiting for an end-quote character. As he put it, "Malformed files are malformed files and should probably fail in some regard, but a hang is obviously undesirable."

Essentially, the CSVLineReader needs to be tweaked in such a way that an informative exception is thrown after some threshold is reached, instead of basically just hanging.

Issue Links

• relates to: CRUNCH-362 Add a CSV File Source