[HIVE-26986] ParallelEdgeFixer adds redundant reduce sink operators

• Type: Sub-task
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 4.0.0-alpha-2
• Labels: hive-4.1.0-must pull-request-available
• Target Version/s: 4.1.0
• Assignee: Seonggon Namgung
• Reporter: Seonggon Namgung
• Created: 25/Jan/23 10:11
• Updated: 02/Dec/24 08:49

Description

A DAG created by OperatorGraph is not equal to the corresponding DAG that is submitted to Tez.

Because of this problem, ParallelEdgeFixer reports a pair of normal edges to a parallel edge.

We observe this problem by comparing OperatorGraph and Tez DAG when running TPC-DS query 71 on 1TB ORC format managed table.

Issue Links

• links to: GitHub Pull Request #3998