[IMPALA-4551] Set limits on size of expression trees

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.8.0
• Fix Version/s: Impala 3.3.0
• Component/s: Backend
• Target Version: Product Backlog
• Assignee: Joe McDonnell
• Reporter: Tim Armstrong
• Created: 29/Nov/16 00:40
• Updated: 10/Aug/24 14:10
• Resolved: 14/Aug/19 23:20

Description

Very large expression trees can cause havoc in various Impala components. I have been experimenting with the attached test that generates large case statements of varying depths and widths, and have been able to hit limits in the frontend (Java OOM) and caused various runaway memory usage problems in the backend (thrift structures, LLVM IR, codegen, etc).

We should set some kind of limit here, either on the number of nodes in the expression trees, or on the size of the query text, and then make sure that we can execute queries of the maximum size end-to-end.

Issue Links

• causes: IMPALA-12652 Limit Length of Completed Queries Insert DML
• is depended upon by: IMPALA-3259 Codegen is not cancellable and can use a lot of CPU and memory
• is duplicated by: IMPALA-3254 query archiving can temporarily take 1g memory for a single 500KB query