[HIVE-28310] Disable hive.optimize.join.disjunctive.transitive.predicates.pushdown by default

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.1.0, 4.0.1
• Component/s: CBO
• Labels: hive-4.0.1-merged hive-4.0.1-must pull-request-available
• Assignee: Stamatis Zampetakis
• Reporter: Stamatis Zampetakis
• Created: 07/Jun/24 09:11
• Updated: 24/Sep/24 02:15
• Resolved: 01/Jul/24 07:55

Description

HIVE-25758 introduced hive.optimize.join.disjunctive.transitive.predicates.pushdown  to conditionally limit some features of the HiveJoinPushTransitivePredicatesRule which are rather unsafe and can lead to Hiveserver2 crashes (OOM, hangs, etc.). 

The property was initially set to true to retain the old behavior and prevent changes in performance for those queries that work fine as is. However, when the property is true there are various known cases/queries that can bring down HS2 completely. When this happens debugging, finding the root cause, and turning off the property may require lots of effort from developers and users.

In this ticket, we propose to disable the property by default and thus limit the optimizations performed by the rule (at least till a complete solution is found for the known problematic cases).

This change favors HS2 stability at the expense of slight performance degradation in certain queries.

Issue Links

• relates to: HIVE-25758 OOM due to recursive application of CBO rules
• links to: GitHub Pull Request #5286