[DRILL-4788] Exporting from Parquet to CSV - commas in strings are not escaped

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.6.0
• Fix Version/s: 1.17.0
• Component/s: Functions - Drill
• Labels: csv csvparser export

Environment

Linux

• Assignee: Unassigned
• Reporter: Richard Patching
• Created: 19/Jul/16 07:33
• Updated: 22/Oct/19 10:20
• Resolved: 22/Oct/19 10:20

Description

When exporting data from Parquet to CSV, if there is a column which contains a comma, the text after the comma gets put into the next column instead of being escaped.

The only work around is to do REGEXP_REPLACE(COLUMN[0], ',',' ') which replaced the comma in the string with a blank space. This is not ideal in terms of keeping a true accurate record of the data we receive.

Issue Links

• relates to: DRILL-6096 Provide mechanisms to specify field delimiters and quoted text for TextRecordWriter