[DRILL-3562] Query fails when using flatten on JSON data where some documents have an empty array

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.1.0
• Fix Version/s: 1.10.0
• Component/s: Storage - JSON
• Assignee: Serhii Harnyk
• Reporter: Philip Deegan
• Reviewer: Rahul Kumar Challapalli
• Created: 27/Jul/15 10:26
• Updated: 05/Apr/17 17:59
• Resolved: 30/Jan/17 20:51

Description

Drill query fails when using flatten when some records contain an empty array

SELECT COUNT(*) FROM (SELECT FLATTEN(t.a.b.c) AS c FROM dfs.`flat.json` t) flat WHERE flat.c.d.e = 'f' limit 1;

Succeeds on
{ "a": { "b": { "c": [ { "d":

{ "e": "f" }

} ] } } }

Fails on
{ "a": { "b":

{ "c": [] }

} }

Error

Error: SYSTEM ERROR: ClassCastException: Cannot cast org.apache.drill.exec.vector.NullableIntVector to org.apache.drill.exec.vector.complex.RepeatedValueVector

Is it possible to ignore the empty arrays, or do they need to be populated with dummy data?

Issue Links

• is duplicated by: DRILL-4578 "children" missing from results of full scan over JSON data
• relates to: DRILL-5396 A flatten query on top of 2 files with one record each causes oversize allocation error randomly
• relates to: DRILL-2217 Trying to flatten an empty list should return an empty result
• links to: GitHub Pull Request #713