Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-4317

Some rules fail to handle Aggregate node if RelFieldTrimmer trims all the fields

    XMLWordPrintableJSON

    Details

      Description

      Consider this query:

      select o_orderkey from 
      (SELECT count(*) cnt_star, count(l_orderkey) cnt_ok FROM lineitem)
      cross join orders
      limit 10
      

       which generates this plan:

      LogicalSort(fetch=[10])
        LogicalProject(o_orderkey=[$2])
          LogicalJoin(condition=[true], joinType=[inner])
            LogicalAggregate(group=[{}], cnt_star=[COUNT()], cnt_ok=[COUNT($0)])
              LogicalProject(l_orderkey=[$0])
                ScanCrel(table=[lineitem], columns=[`l_orderkey`, `l_partkey`, `l_suppkey`, `l_linenumber`, `l_quantity`, `l_extendedprice`, `l_discount`, `l_tax`, `l_returnflag`, `l_linestatus`, `l_shipdate`, `l_commitdate`, `l_receiptdate`, `l_shipinstruct`, `l_shipmode`, `l_comment`])
            ScanCrel(table=[orders], columns=[`o_orderkey`, `o_custkey`, `o_orderstatus`, `o_totalprice`, `o_orderdate`, `o_orderpriority`, `o_clerk`, `o_shippriority`, `o_comment`])
      

      and after we apply RelFieldTrimmer:

      LogicalSort(fetch=[10])
        LogicalProject(o_orderkey=[$0])
          LogicalJoin(condition=[true], joinType=[inner])
            LogicalAggregate(group=[{}])
              LogicalProject(l_orderkey=[$0])
                ScanCrel(table=[lineitem], columns=[`l_orderkey`])
            ScanCrel(table=[orders], columns=[`o_orderkey`])
      

       

      We see that the LogicalAggregate has no groups/functions after applying the trimmer. When we apply ProjectJoinTransposeRule or LoptOptimizeJoinRule after trimming, we are seeing these exceptions:

      For LoptOptimizeJoinRule:

        (java.lang.ArrayIndexOutOfBoundsException) 0
          com.google.common.collect.RegularImmutableList.get():75
          org.apache.calcite.rel.metadata.RelMdColumnOrigins.getColumnOrigins():77
          sun.reflect.NativeMethodAccessorImpl.invoke0():-2
          sun.reflect.NativeMethodAccessorImpl.invoke():62
          sun.reflect.DelegatingMethodAccessorImpl.invoke():43
          java.lang.reflect.Method.invoke():498
          org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$1$1.invoke():178
          com.sun.proxy.$Proxy175.getColumnOrigins():-1
          sun.reflect.GeneratedMethodAccessor144.invoke():-1
          sun.reflect.DelegatingMethodAccessorImpl.invoke():43
          java.lang.reflect.Method.invoke():498
          org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke():139
          com.sun.proxy.$Proxy175.getColumnOrigins():-1
          sun.reflect.GeneratedMethodAccessor144.invoke():-1
          sun.reflect.DelegatingMethodAccessorImpl.invoke():43
          java.lang.reflect.Method.invoke():498
          com.sun.proxy.$Proxy175.getColumnOrigins():-1
          sun.reflect.GeneratedMethodAccessor144.invoke():-1
          sun.reflect.DelegatingMethodAccessorImpl.invoke():43
          java.lang.reflect.Method.invoke():498
          org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke():139
          com.sun.proxy.$Proxy175.getColumnOrigins():-1
          sun.reflect.GeneratedMethodAccessor144.invoke():-1
          sun.reflect.DelegatingMethodAccessorImpl.invoke():43
          java.lang.reflect.Method.invoke():498
          com.sun.proxy.$Proxy175.getColumnOrigins():-1
          sun.reflect.GeneratedMethodAccessor144.invoke():-1
          sun.reflect.DelegatingMethodAccessorImpl.invoke():43
          java.lang.reflect.Method.invoke():498
          org.apache.calcite.rel.metadata.RelMetadataQuery$MetadataInvocationHandler.invoke():110
          com.sun.proxy.$Proxy175.getColumnOrigins():-1
          org.apache.calcite.rel.metadata.RelMetadataQuery.getColumnOrigins():293
          org.apache.calcite.rel.metadata.RelMetadataQuery.getTableOrigin():348
          org.apache.calcite.rel.rules.LoptOptimizeJoinRule.getSimpleFactors():377
          org.apache.calcite.rel.rules.LoptOptimizeJoinRule.findRemovableSelfJoins():297
          org.apache.calcite.rel.rules.LoptOptimizeJoinRule.onMatch():126
          org.apache.calcite.plan.AbstractRelOptPlanner.fireRule():317
          org.apache.calcite.plan.hep.HepPlanner.applyRule():556
          org.apache.calcite.plan.hep.HepPlanner.applyRules():415
          org.apache.calcite.plan.hep.HepPlanner.executeInstruction():280
          org.apache.calcite.plan.hep.HepInstruction$RuleCollection.execute():74
          org.apache.calcite.plan.hep.HepPlanner.executeProgram():211
          org.apache.calcite.plan.hep.HepPlanner.findBestExp():198
      

      For ProjectJoinTransposeRule:

        (java.lang.ArrayIndexOutOfBoundsException) 0
          com.google.common.collect.RegularImmutableList.get():75
          org.apache.calcite.rel.rules.PushProjector.createProjectRefsAndExprs():477
          org.apache.calcite.rel.rules.ProjectJoinTransposeRule.onMatch():138
          org.apache.calcite.plan.AbstractRelOptPlanner.fireRule():317
          org.apache.calcite.plan.hep.HepPlanner.applyRule():556
          org.apache.calcite.plan.hep.HepPlanner.applyRules():415
          org.apache.calcite.plan.hep.HepPlanner.executeInstruction():280
          org.apache.calcite.plan.hep.HepInstruction$RuleCollection.execute():74
          org.apache.calcite.plan.hep.HepPlanner.executeProgram():211
          org.apache.calcite.plan.hep.HepPlanner.findBestExp():198
      

      I think we should either fix the trimmer, or the rules to handle these cases.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                Rafay Rafay A
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 4.5h
                  4.5h