[IMPALA-4080] Share codegen work between fragment instances

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.8.0
• Fix Version/s: Impala 4.0.0
• Component/s: Backend
• Labels: codegen multithreading
• Epic Link: Impala Multi-threaded query execution
• Target Version: Impala 4.0.0
• Assignee: Bikramjeet Vig
• Reporter: Tim Armstrong
• Created: 06/Sep/16 19:40
• Updated: 09/Mar/23 17:28
• Resolved: 06/Apr/20 17:56

Description

With multithreading, we should find a way to share codegen modules between fragment instances to avoid redundant compilation, optimisation, etc.

Issue Links

• breaks: IMPALA-6262 Crash Impalad [ DataSink::Create fail which cause profile nullptr ]
• incorporates: IMPALA-5192 Avoid hard coding pointer to the tuple pool into generated IR of Tuple::CodegenMaterializeExprs()
• is blocked by: IMPALA-4008 Generate codegen module without operator/query-specific constants
• is blocked by: IMPALA-4192 Pull all expressions in a fragment into QueryState
• is blocked by: IMPALA-4108 Move cached set_is_null/is_null/etc functions from SlotDescriptor to LlvmCodeGen
• is related to: IMPALA-5356 Move Expr's fragment local states from ExprContext to Expr
• relates to: IMPALA-9216 Pull the rest of the fragment level state out of the exec nodes after IMPALA-4192

Sub-Tasks

• Pull all expressions in a fragment into QueryState: Resolved
• Remove embedded pointer references in handcrafted codegen code: Resolved