[HIVE-26524] Use Calcite to remove sections of a query plan known never produces rows

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-2
• Component/s: CBO
• Labels: pull-request-available
• Target Version/s: 4.0.0
• Assignee: Krisztian Kasa
• Reporter: Krisztian Kasa
• Created: 08/Sep/22 14:44
• Updated: 25/Apr/23 14:38
• Resolved: 07/Oct/22 03:46

Description

Calcite has a set of rules to remove sections of a query plan known never produces any rows. In some cases the whole plan can be removed. Such plans are represented with a single Values operators with no tuples. ex.:

select y + 1 from (select a1 y, b1 z from t1 where b1 > 10) q WHERE 1=0

HiveValues(tuples=[[]])

Other cases when plan has outer join or set operators some branches can be replaced with empty values moving forward in some cases the join/set operator can be removed

select a2, b2 from t2 where 1=0
union
select a1, b1 from t1

HiveAggregate(group=[{0, 1}])
  HiveTableScan(table=[[default, t1]], table:alias=[t1])

Issue Links

• is related to: CALCITE-5282 JdbcValues should add CAST on NULL values
• is related to: HIVE-26582 Cartesian join fails if the query has an empty table when cartesian product edge is used
• is related to: HIVE-24466 insert queries should not launch job when condition in the query would output 0 rows
• is related to: HIVE-26559 Skip unnecessary get all partition operations when where condition with 1=0 in CBO.
• is related to: HIVE-26618 Add setting to turn on/off removing sections of a query plan known never produces rows
• is related to: HIVE-27253 Support array type when query does not produce any rows
• is related to: HIVE-17342 Where condition with 1=0 should be treated similar to limit 0
• is required by: HIVE-25953 Drop HiveRelMdPredicates::getPredicates(Join...) to use that of RelMdPredicates
• relates to: HIVE-26817 Set column names in result schema when plan has Values root
• links to: GitHub Pull Request #3588
• links to: GitHub Pull Request #3753