[HIVE-26968] Wrong results when shared work optimizer merges TS operator with different DPP edges

• Type: Sub-task
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 4.0.0-alpha-2
• Fix Version/s: 4.0.0-beta-1
• Labels: hive-4.0.0-must pull-request-available
• Target Version/s: 4.0.0
• Assignee: Seonggon Namgung
• Reporter: Seonggon Namgung
• Created: 20/Jan/23 07:48
• Updated: 15/Aug/23 06:33
• Resolved: 08/Jun/23 07:37

Description

SharedWorkOptimizer merges TableScan operators that have different DPP parents, which leads to the creation of semantically wrong query plan.

In our environment, running TPC-DS query64 on 1TB Iceberg format table returns no rows  because of this problem. (The correct result has 7094 rows.)

We use hive.optimize.shared.work=true, hive.optimize.shared.work.extended=true, and hive.optimize.shared.work.dppunion=false to reproduce the bug.

Issue Links

• is duplicated by: HIVE-27134 SharedWorkOptimizer merges TableScan operators that have different DPP parents
• links to: GitHub Pull Request #3981