[DRILL-4341] Fails to parse string literals containing escaped quotes

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.0
• Fix Version/s: 1.8.0
• External issue ID: DRILL-2669
• Assignee: Unassigned
• Reporter: Oscar Morante
• Created: 02/Feb/16 19:09
• Updated: 07/Aug/16 10:36
• Resolved: 07/Aug/16 10:36

Description

In some situations, Drill fails to parse strings with escaped single quotes. Just changing the last line of this query makes the problem appear and disappear (without touching the string it's complaining about):

select
  'yyyy-MM-dd''T''HH:mm:ss.SSS''Z'''
from
  s3; --> FAILS
  --s3.`2015`; --> FAILS
  --s3.`2015/11`; --> FAILS
  --s3.`2015/11/3`; --> WORKS
  --s3.`2015/11/3/10-1072045-1612661.json.gz`; --> WORKS
  --(select * from s3 limit 1); --> WORKS
  --(select * from s3.`2015` limit 1); --> WORKS
  --(select * from s3.`2015/11` limit 1); --> WORKS

This is the error when it fails:

ExampleExceptionFormatter: exception message was: SYSTEM ERROR: ExpressionParsingException: Expression has syntax error! line 1:12:missing EOF at 'T'

Fragment 1:1

This is very important when dealing with json files containing dates encoded as iso-8601 strings. My current workaround is something like this:

select to_timestamp(regexp_replace(`timestamp`, '[TZ]', ''),
                    'yyyy-MM-ddHH:mm:ss.SSS')
from s3;