[HIVE-24812] Disable sharedworkoptimizer remove semijoin by default

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Labels: pull-request-available
• Assignee: Zoltan Haindrich
• Reporter: Zoltan Haindrich
• Created: 23/Feb/21 12:48
• Updated: 17/Nov/22 08:48
• Resolved: 11/Mar/21 09:52

Description

SJ removal backfired a bit when I was testing stuff - because of the additional opportunities paralleledges may enable ; because it will increased the shuffled memory amount and/or even make MJ broadcast inputs larger

set hive.optimize.shared.work.semijoin=false by default for now

right now it's better to leave dppunion to pick up these cases instead of removing the SJ fully - after HIVE-24376 we might enable it back

Issue Links

• relates to: HIVE-24376 SharedWorkOptimizer may retain the SJ filter condition during RemoveSemijoin mode
• links to: GitHub Pull Request #2006