[DRILL-3323] Flatten planning rule creates unneeded copy of the list being flattened, causes executuion/allocation issues with large lists

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: Future
• Component/s: Execution - Flow
• Assignee: Unassigned
• Reporter: Jason Altekruse
• Created: 19/Jun/15 17:12
• Updated: 01/Oct/15 16:37

Description

The planning rule for flatten was written to not only handle the flatten operator, but it also was designed to address some shortcomings in expression evaluation involving complex types. The rule currently plans inefficiently to try to cover some of these more advanced cases, but there is not thorough test coverage to even demonstrate the benefits of it. We should disable a particular behavior of copying complex data and extra time when it is not needed, because it is causing flatten queries to fail with allocation issues.

Issue Links

• relates to: DRILL-2851 Memory LEAK - FLATTEN function fails when input array has 99,999 integer type elements