[ASTERIXDB-3429] Allow configurable escape character for delimited text parser

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.9.10
• Component/s: EXT - External data
• Labels: triaged
• Assignee: Ian Maxon
• Reporter: Ian Maxon
• Created: 12/Jun/24 23:23
• Updated: 21/Jun/24 19:24
• Resolved: 21/Jun/24 17:02

Description

The current CSV/Delimited Text parser supports RFC4180-style escaping rules. In brief, this means that string fields can be escaped by surrounding them in " , and " itself can be escaped as ""  .

However, many CSV files do not follow this convention. For example, this row using C-style escaping of quotes would be invalid following RFC4180:

 

1,"The \"quick\" fox, jumped over the lazy dog "

The CSV parser should allow the escape character to be configurable, to allow parsing of these nonstandard, but common examples.