[IMPALA-4192] Pull all expressions in a fragment into QueryState

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.8.0
• Fix Version/s: Impala 4.0.0
• Component/s: Backend
• Labels: codegen
• Assignee: Bikramjeet Vig
• Reporter: Michael Ho
• Created: 22/Sep/16 23:56
• Updated: 06/Apr/20 17:59
• Resolved: 06/Apr/20 17:57

Description

Currently, Exprs in a fragment are scattered through the TPlanNode in a fragment. Expr is supposed to be shared by all threads but they are always accessed through thread-private state ExprContext. With multi-threading, an Expr in a fragment and its generated IR function should be shared among all instances of it.

As part of IMPALA-4080, all Expr in a fragment will be moved to query-wide state instead of having multiple copies of them in the exec nodes of each instances. In addition, moving Expr to query-wide state prevents the mistakes of baking in thread private states (e.g. ExprContext pointers) into the generated IR code for an Expr.

Issue Links

• blocks: IMPALA-4080 Share codegen work between fragment instances
• breaks: IMPALA-5551 test_failpoint.py fails in legacy join and agg tests
• breaks: IMPALA-5553 expr-test fails in release builds
• breaks: IMPALA-6184 Check failed: !initialized_ || closed_
• breaks: IMPALA-9309 Crash in failure/test_failpoints.py with PREPARE:FAIL debug option
• relates to: IMPALA-4743 Evaluate constant arguments to ScalarFnCall once during initialization
• relates to: IMPALA-4356 Automatically codegen expressions with any root Expr node