[IMPALA-12657] Improve ProcessingCost of ScanNode and NonGroupingAggregator

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 4.3.0
• Fix Version/s: Impala 4.4.0
• Component/s: Frontend
• Target Version: Impala 4.4.0
• Epic Color: ghx-label-1
• Assignee: David Rorke
• Reporter: Riza Suminto
• Created: 19/Dec/23 21:56
• Updated: 22/May/24 18:12
• Resolved: 22/May/24 18:12

Description

Several benchmark run measuring Impala scan performance indicates some costing improvement opportunity around ScanNode and NonGroupingAggregator.

profile_1f4d7a679a3e12d5_4223115700000000.txt shows an example of simple count query.

Key takeaway:

1. There is a strong correlation between total materialized bytes (row-size * cardinality) with total materialized tuple time per fragment. Row materialization cost should be adjusted to be based on this row-sized instead of equal cost per scan range.
2. NonGroupingAggregator should have much lower cost that GroupingAggregator. In example above, the cost of NonGroupingAggregator dominates the scan fragment even though it only does simple counting instead of hash table operation.

Issue Links

• is related to: IMPALA-11972 Factor in row width during ProcessingCost calculation.