[HIVE-25870] Avoid simplification in HivePointLookupOptimizerRule, convert only

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 4.0.0
• Component/s: CBO
• Assignee: Unassigned
• Reporter: Alessandro Solimando
• Created: 17/Jan/22 16:12
• Updated: 17/Jan/22 16:18

Description

HivePointLookupOptimizerRule has been introduced to improve simplifications and improve statistics/estimations (see https://issues.apache.org/jira/browse/HIVE-11424?focusedCommentId=15197407&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15197407).

At the time, RexSimplify could not simplify such OR/AND expressions (as reported in the JIRA above):

simplify(unknown as unknown): AND(true, OR(=(?0.int0, 1), =(?0.int0, 2), =(?0.int0, 3)), OR(AND(true, =(?0.int0, 1)), AND(true, =(?0.int0, 2))))

For Calcite <= 1.25, simplification is still missed:

Expected: "OR(?0.int0=1, ?0.int0=2)"
     but: was "AND(OR(=(?0.int0, 1), =(?0.int0, 2), =(?0.int0, 3)), OR(=(?0.int0, 1), =(?0.int0, 2)))"

From Calcite >= 1.26, the simplifications happens:

Expected: "OR(?0.int0=1, ?0.int0=2)"
     but: was "SEARCH(?0.int0, Sarg[1, 2])"

For this reason, once we upgrade to Calcite >= 1.26, we could drop all the simplifications within the rule, just keep the OR -> IN conversion, and move the rule to the very last planning stage.

Issue Links

• is related to: HIVE-25852 Introduce IN clauses at the very end of query planning