[HIVE-25766] java.util.NoSuchElementException in HiveFilterProjectTransposeRule if predicate has no InputRef

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0-alpha-1
• Component/s: CBO, Query Planning
• Labels: pull-request-available
• Assignee: Alessandro Solimando
• Reporter: Alessandro Solimando
• Created: 02/Dec/21 18:31
• Updated: 17/Nov/22 08:47
• Resolved: 17/Feb/22 13:43

Description

The issue can be reproduced with the following query:

create table test1 (s string);
create table test2 (m string);

EXPLAIN CBO SELECT c.m
FROM (
  SELECT substr(from_unixtime(unix_timestamp(), 'yyyy-MM-dd'), 1, 1) as m
  FROM test1
  WHERE substr(from_unixtime(unix_timestamp(), 'yyyy-MM-dd'), 1, 1) = '2') c
JOIN test2 d ON c.m = d.m;

It fails with the following exception:

 java.util.NoSuchElementException
    at java.util.HashMap$HashIterator.nextNode(HashMap.java:1447)
    at java.util.HashMap$KeyIterator.next(HashMap.java:1469)
    at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.check(HiveFilterProjectTransposeRule.java:348)
    at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:306)
    at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:303)
    at org.apache.calcite.rel.SingleRel.childrenAccept(SingleRel.java:72)
    at org.apache.calcite.rel.RelVisitor.visit(RelVisitor.java:44)
    at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:316)
    at org.apache.calcite.rel.RelVisitor.go(RelVisitor.java:61)
    at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule.isRedundantIsNotNull(HiveFilterProjectTransposeRule.java:276)
    at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule.onMatch(HiveFilterProjectTransposeRule.java:191)

The current implementation, while checking if the predicate to be transposed is redundant or not, it expects at least one InputRef, but the predicate can have none as in this case.

Issue Links

• links to: GitHub Pull Request #2839