[HIVE-27831] Set hive.cbo.fallback.strategy to NEVER by default

• Type: Task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: CBO
• Labels: pull-request-available
• Assignee: Stamatis Zampetakis
• Reporter: Stamatis Zampetakis
• Created: 30/Oct/23 13:40
• Updated: 10/Oct/24 10:35

Description

The hive.cbo.fallback.strategy property defines when Hive fallbacks to legacy optimizer if an error occurs during the CBO phase.

At the moment the default value is CONSERVATIVE, which is the backward compatible option, and automatically fallbacks to legacy optimizer when certain errors occur.

The legacy optimizer is (hive.cbo.enable=false) is gonna soon be officially deprecated (HIVE-27830); unofficially it was treated as such for a long time now.

To reduce maintenance cost and improve CBO coverage and stability we should NEVER fallback to the legacy optimizer after a CBO error.

NEVER should be the default behavior in newer releases; users can still set the property to CONSERVATIVE as a temporary workaround till the CBO error is addressed.

Issue Links

• is related to: HIVE-28571 Basic UNIONTYPE support in CBO
• relates to: HIVE-28193 Skip CBO fallback when EXCEPT/INTERSECT/QUALIFY are used
• relates to: HIVE-24601 Control CBO fallback behavior via property
• relates to: HIVE-27830 Deprecate hive.cbo.enable property
• links to: GitHub Pull Request #4836
• links to: GitHub Pull Request #5284

Sub-Tasks

• Support ambiguous column references when CBO fallback is disabled: Resolved
• Support non-boolean WHERE conditions in CBO: Resolved
• Support select alias in the having clause for CBO: Resolved
• Skip CBO for UNIQUEJOIN and TABLESAMPLE: Resolved
• Support ARRAY field access in CBO: Open
• Skip CBO for union type: Resolved
• Basic UNIONTYPE support in CBO: Resolved