[HIVE-28488] Merge adjacent union distinct

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.1.0
• Component/s: Logical Optimizer
• Labels: hive-4.1.0-must pull-request-available
• Target Version/s: 4.1.0
• Assignee: Seonggon Namgung
• Reporter: Seonggon Namgung
• Created: 29/Aug/24 11:09
• Updated: 13/Nov/24 13:40
• Resolved: 13/Nov/24 13:40

Description

Current Hive compiles
"SELECT * FROM TBL1 UNION SELECT * FROM TBL2 UNION SELECT * FROM TBL3"
to

TS - GBY - RS
TS - GBY - RS - GBY - RS
           TS - GBY - RS - GBY 

This can be optimized as follows:

TS - GBY - RS
TS - GBY - RS
TS - GBY - RS - GBY 

Please check out the attached slides for detailed explanation and feel free to ask any questions or share suggestions. Also, it would be glad if one can share about better location of this optimization (e.g. SemanticAnalyzer, Calcite, etc.).

Issue Links

• links to: GitHub Pull Request #5423