[DRILL-2221] CTAS (JSON) creates unreadable files when writing empty arrays

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Component/s: Storage - JSON
• Assignee: Jason Altekruse
• Reporter: Abhishek Girish
• Created: 12/Feb/15 02:16
• Updated: 07/May/15 19:41
• Resolved: 05/May/15 01:45

Description

Source JSON file:

{
"a": "A1",
"b": [ "B1", "B2" ],
"c": [ "C1", "C2" ]
}
{
"a": "A2",
"b": [ "B3" ],
"c": []
}

> select * from `file.json` ;
--------------------------------

a

b

c

--------------------------------

A1	["B1","B2"]	["C1","C2"]
A2	["B3"]	[]

--------------------------------
2 rows selected (0.039 seconds)

>alter session set `store.format` = 'json';

> create table file_json as select * from `file.json` ;
[Success]

Created file contents:

{
"a" : "A1",
"b" : [ "B1", "B2" ],
"c" : [ "C1", "C2" ]
}

{ "a" : "A2", "b" : [ "B3" ], "c" } This file cannot be read by Drill as the object "c" is not in key:value format: > select * from file_json ; Query failed: Query stopped., Unexpected character ('}

' (code 125)): was expecting a colon to separate field name and value
at [Source: org.apache.drill.exec.vector.complex.fn.JsonReader@67b4040a; line: 8, column: 2] [ 64bb2b5d-5d83-4ae3-a7dc-7a7e44b28bf5 on abhi8.qa.lab:31010 ]
Error: exception while executing query: Failure while executing query. (state=,code=0)

Log attached.