[CALCITE-3457] RexSimplify incorrectly simplifies IS NOT NULL operator with ITEM call

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.22.0
• Fix Version/s: 1.22.0
• Labels: pull-request-available
• Assignee: Danny Chen
• Reporter: Vova Vysotskyi
• Created: 29/Oct/19 17:24
• Updated: 16/Nov/20 22:17
• Resolved: 04/Nov/19 02:43

Description

In CALCITE-3390 ITEM was marked with Policy.ANY strong policy, but according to its JavaDoc, the result may be null if and only if at least one of its arguments is null. This statement was used in RexSimplify.simplifyIsNotNull() method, so t1.c_nationkey[0] is not null will be simplified to IS NOT NULL($0) which is wrong, since array may be empty, or index may be less than the size of the array.

Unit test which helps to reproduce this issue:

  @Test public void testSimplifyItemIsNotNull() {
    String query = "select * from sales.customer as t1 where t1.c_nationkey[0] is not null";

    sql(query)
        .withTester(t -> createDynamicTester())
        .withRule(ReduceExpressionsRule.FILTER_INSTANCE)
        .check();
  }

Returns plan with incorrectly simplified ITEM expression:

LogicalProject(**=[$1])
  LogicalFilter(condition=[IS NOT NULL($0)])
    LogicalTableScan(table=[[CATALOG, SALES, CUSTOMER]])

But the initial intention of CALCITE-3390 was to allow pushing ITEM expression to the right input of left-outer-join.

I propose to add a new element to the Policy which will have a relaxed condition - expression is null if at least one of its arguments is null.

Issue Links

• relates to: CALCITE-3500 Validator infers incorrect type for "IS NOT DISTINCT FROM"
• relates to: CALCITE-4377 Fix the return nullability inference during rex simplification
• links to: GitHub Pull Request #1551