[HIVE-19601] Unsupported Post join function 'IS NOT DISTINCT FROM'

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Target Version/s: 3.1.0
• Assignee: Unassigned
• Reporter: Slim Bouguerra
• Created: 17/May/18 22:23
• Updated: 17/May/18 22:23

Description

As part of trying to use the Calcite rule

 org.apache.calcite.rel.rules.AggregateExpandDistinctAggregatesRule#JOIN 

i got the following Calcite plan

2018-05-17T09:26:02,781 DEBUG [80d6d405-ed78-4f60-bd93-b3e08e424f73 main] translator.PlanModifierForASTConv: Final plan after modifier
 HiveProject(_c0=[$1], _c1=[$2])
  HiveProject(zone=[$0], $f1=[$1], $f2=[$3])
    HiveJoin(condition=[IS NOT DISTINCT FROM($0, $2)], joinType=[inner], algorithm=[none], cost=[not available])
      HiveProject(zone=[$0], $f1=[$1])
        HiveAggregate(group=[{0}], agg#0=[count($1)])
          HiveProject(zone=[$0], interval_marker=[$1])
            HiveAggregate(group=[{0, 1}])
              HiveProject(zone=[$3], interval_marker=[$1])
                HiveTableScan(table=[[druid_test_dst.test_base_table]], table:alias=[test_base_table])
      HiveProject(zone=[$0], $f1=[$1])
        HiveAggregate(group=[{0}], agg#0=[count($1)])
          HiveProject(zone=[$0], dim=[$1])
            HiveAggregate(group=[{0, 1}])
              HiveProject(zone=[$3], dim=[$4])
                HiveTableScan(table=[[druid_test_dst.test_base_table]], table:alias=[test_base_table])

I run into this issue

 
2018-05-17T09:26:02,876 ERROR [80d6d405-ed78-4f60-bd93-b3e08e424f73 main] parse.CalcitePlanner: CBO failed, skipping CBO.
org.apache.hadoop.hive.ql.parse.SemanticException: Line 0:-1 Invalid function 'IS NOT DISTINCT FROM'
        at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:1069) ~[hive-exec-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:1464) ~[hive-exec-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) ~[hive-exec-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105) ~[hive-exec-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]