[IMPALA-5444] Asynchronous code generation

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Implemented
• Fix Version/s: Impala 4.0.0
• Component/s: Backend
• Labels: codegen
• Target Version: Product Backlog
• Epic Color: ghx-label-7
• Assignee: Daniel Becker
• Reporter: Michael Ho
• Created: 06/Jun/17 18:44
• Updated: 27/Jul/23 21:14
• Resolved: 16/Jul/20 10:57

Description

Currently, codegen happens during the preparation phase of a query fragment. In other words, the query fragment cannot start running until the code generation is complete. There are queries in which the code generation time is taking a huge amount of time. While we should disable codegen in some exec nodes if we can accurately estimate in the planner that running without codegen will be better off (e.g. number of rows to process is relatively small), we will still pay the price if say the stats is stale or the estimation is off.

With async codegen, the idea is that we should run the code generation in a separate thread so that codegen is not on the critical path of the query execution. Once codegen completes for a fragment, we can atomically swap the function pointers of compiled functions embedded in the exec nodes. The exec nodes all currently support falling back to interpretation if the codegend functions don't exist anyway (i.e. the pointer to the compiled function is NULL). In some cases, it can occur that the query may run to completion before codegen completes. Once IMPALA-3259 is fixed (if feasible), we should be able to cancel the codegen execution.

Another thing to note is that we should be able to bound the codegen work to a set of threads in thread pool so as to control the CPU and memory resources consumed by codegen.

Another potential extension of this decoupling is IMPALA-9660.

Issue Links

• is depended upon by: IMPALA-9660 Distributed codegen
• is depended upon by: IMPALA-11460 Enable ASYNC CODEGEN by default
• is related to: IMPALA-2651 codegen overhead can be high