[DRILL-1249] JSON select with "LIMIT 1" throws exception when NULL exists in array

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.4.0
• Fix Version/s: 0.5.0
• Component/s: Storage - JSON

Environment

5-node Hadoop cluster (MapR 3.1.1)

• Assignee: DrillCommitter
• Reporter: Andy Pernsteiner
• Due: 11/Aug/14
• Created: 03/Aug/14 18:19
• Updated: 07/May/15 21:34
• Resolved: 12/Aug/14 16:24

Description

This may be related to DRILL-1239

With a JSON file that has the following contents:

{"trans_id":0,"date":"10/24/2012","time":"06:14:07","cust_id":3,"device":"IOS5","state":"nj","camp_id":4,"keywords":["the"],"prod_id":[],"purch_flag":"true"}
{"trans_id":1,"date":"06/07/2013","time":"12:29:37","cust_id":111464,"device":"IOS5","state":"va","camp_id":4,"keywords":[],"prod_id":[78,116,18,508,28,309,149,80,360,6,28,52,235],"purch_flag":"false"}
{"trans_id":2,"date":"05/16/2013","time":"04:55:41","cust_id":2817,"device":"IOS5","state":"sc","camp_id":10,"keywords":[],"prod_id":[179,247],"purch_flag":"false"}
{"trans_id":3,"date":"06/03/2013","time":"08:14:23","cust_id":14085,"device":"IOS6","state":"oh","camp_id":6,"keywords":["steal"],"prod_id":[510,142,18],"purch_flag":"false"}
{"trans_id":4,"date":"04/10/2013","time":"06:32:31","cust_id":1044,"device":"IOS6","state":"il","camp_id":16,"keywords":[],"prod_id":[25,28,316],"purch_flag":"false"}
{"trans_id":5,"date":"07/27/2013","time":"10:02:24","cust_id":13,"device":"IOS5","state":"oh","camp_id":4,"keywords":["town","you"],"prod_id":[4,1,79,449],"purch_flag":"false"}

Performing a select w/ a LIMIT throws an error if the LIMIT lines up with a null value (I think):

0: jdbc:drill:> select * from click.`json_small`.`small.json` limit 1;
Error: exception while executing query: null (state=,code=0)

If I re-run the query with a larger limit I don't hit the error:

0: jdbc:drill:> select * from click.`json_small`.`small.json` limit 2;
+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+
|  trans_id  |    date    |    time    |  cust_id   |   device   |   state    |  camp_id   |  keywords  | purch_flag |  prod_id   |
+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+
| 0          | 10/24/2012 | 06:14:07   | 3          | IOS5       | nj         | 4          | ["the"]    | true       | []         |
| 1          | 06/07/2013 | 12:29:37   | 111464     | IOS5       | va         | 4          | []         | false      | [78,116,18,508,28,309,149,80,360,6,28,52,235] |
+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+
2 rows selected (0.223 seconds)

Note that the first record contains a null value for the prod_id array.