[DRILL-1712] Quoted CSV parsing

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.6.0
• Fix Version/s: Future

Environment

MapR 4.0.1 M5

• Assignee: Paul Rogers
• Reporter: Hari Sekhon
• Created: 14/Nov/14 11:39
• Updated: 19/Jun/17 00:15

Description

When querying CSV files Drill doesn't handle quoted CSV files properly and includes the quotes in the data. The directory /tmp/hari in MapR-FS has two simple CSV files, one quoted, one not quoted so you can see the difference.

0: jdbc:drill:> select * from dfs.`/tmp/hari` limit 10;
+------------+
|  columns   |
+------------+
| ["1","2","3"] |
| ["4","5","6"] |
| ["7","8","9"] |
| ["\"1\"","\"2\"","\"3\""] |
| ["\"4\"","\"5\"","\"6\""] |
| ["\"7\"","\"8\"","\"9\""] |
+------------+
6 rows selected (0.238 seconds)

 cat hari/hari.csv
1,2,3
4,5,6
7,8,9
cat hari/hari2.csv
"1","2","3"
"4","5","6"
"7","8","9"

It shouldn't be including the quotes as data, they're just containers to the data.

This is related to DRILL-950 but is not the same issue.

Regards,

Hari Sekhon
http://www.linkedin.com/in/harisekhon