[IMPALA-12200] Cap stats NDV from SetOperationStmt.createMetadata

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 4.3.0
• Fix Version/s: Impala 4.3.0
• Component/s: Frontend
• Epic Color: ghx-label-3
• Assignee: Riza Suminto
• Reporter: Riza Suminto
• Created: 09/Jun/23 20:05
• Updated: 23/Jun/23 01:09
• Resolved: 22/Jun/23 05:49

Description

Union operator will create merged ColumnStats at SetOperationStmt.createMetadata where it adds all ColumnStats from its input children. One of the stats being accumulated is NDV (num distinct value). There is an opportunity to lower the resulting NDV if all source expression is referring to the same column. This lower NDV can benefit Aggregation node on top of the Union node because it can lower cardinality and memory estimate of the Aggregation node.