[HIVE-16867] Extend shared scan optimizer to reuse computation from other operators

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: Physical Optimizer
• Labels: TODOC3.0
• Target Version/s: 3.0.0
• Assignee: Jesús Camacho Rodríguez
• Reporter: Jesús Camacho Rodríguez
• Created: 09/Jun/17 11:34
• Updated: 27/Feb/24 22:23
• Resolved: 22/Jun/17 15:13

Description

Follow-up of the work in HIVE-16602.

HIVE-16602 introduced an optimization that identifies scans on input tables that can be merged so the data is read only once.
This extension to that rule allows to reuse the computation that is done in the work containing those scans. In particular, we traverse both parts of the plan upstream and reuse the operators if possible.
Currently, the optimizer will not go beyond the output edge(s) of that work. Follow-up extensions might remove this limitation.

Issue Links

• is related to: HIVE-18361 Extend shared work optimizer to reuse computation beyond work boundaries
• is related to: HIVE-16945 Add method to compare Operators
• is related to: HIVE-17542 Make HoS CombineEquivalentWorkResolver Configurable
• relates to: HIVE-16602 Implement shared scans with Tez
• links to: RB link