[DRILL-5396] A flatten query on top of 2 files with one record each causes oversize allocation error randomly

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.10.0, 1.11.0
• Component/s: Execution - Data Types, Storage - JSON
• Assignee: Unassigned
• Reporter: Rahul Kumar Challapalli
• Created: 29/Mar/17 23:14
• Updated: 19/May/17 09:54

Description

git.commit.id.abbrev=38ef562

As part of verifying DRILL-3562, I came up with the below 2 files

File 1:

{ "a": { "b": { "c": [] } } }

File 2:

{ "a": { "b": { "c": [1] } } }

Now the below query work on individual files, however when I run the query on a directory containing both the files, I randomly hit the below error

select FLATTEN(t.a.b.c) AS c from dfs.`/drill/testdata/json_kvgenflatten/drill3562` t;
Error: SYSTEM ERROR: OversizedAllocationException: Unable to expand the buffer. Max allowed buffer size is reached.

Fragment 0:0

[Error Id: e556243b-7ad6-4131-81f7-e8b225c9c8bc on qa-node182.qa.lab:31010] (state=,code=0)

This could be related to the fix for DRILL-3562

Issue Links

• is related to: DRILL-3562 Query fails when using flatten on JSON data where some documents have an empty array
• is related to: DRILL-4824 Null maps / lists and non-provided state support for JSON fields. Numeric types promotion.