Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1018

SortJoinTransposeRule not firing due to getMaxRowCount(RelSubset) returning null

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.6.0
    • Component/s: None
    • Labels:
      None

      Description

      getMaxRowCount(RelSubset) defaults to getMaxRowCount(RelNode) and returns null, so if the Join rel has its outer child rel as a RelSubset, the SortJoinTransposeRule will fail the checkInputForCollationAndLimit check and will not proceed to push down a limit through join.
      Before fix of CALCITE-995 and CALCITE-987, getMaxRowCount(RelSubset) would return positive infinity, which would cause similar situation but an opposite effect as firing the rule infinitely when the join's outer child is a RelSubset.
      Neither of the above situation was reflected in the test cases for SortJoinTransposeRule in Calcite, since the RelSubset condition was not covered by RelOptRule test which uses HepPlanner running with just one or no more than a couple of rules together.
      Basically we need a more accurate way to get max row count for RelSubset (similarly for HexVertex as well), otherwise checkInputForCollationAndLimit would either always fail or always succeed for a Limit over a Join over a RelSubset. But I assume, to be real accurate, we'd have to introduce a similar mechanism to one that computes bestCost in RelSubset, which I doubt would be worth it.
      Another way, which might seem a little ugly, is to add something like "isSortPushedThrough()" in Sort rel, similar to "isSemiJoinDone()" in Join, in order to avoid the rule being fired infinitely.
      In my Phoenix project, I applied a temporary fix, but it proved to work:

      public class PhoenixRelMdMaxRowCount {
          public static final RelMetadataProvider SOURCE =
                  ReflectiveRelMetadataProvider.reflectiveSource(
                      BuiltInMethod.MAX_ROW_COUNT.method, new PhoenixRelMdMaxRowCount());
      
          private PhoenixRelMdMaxRowCount() {
          }
      
          public Double getMaxRowCount(RelSubset rel) {
              for (RelNode node : rel.getRels()) {
                  if (node instanceof Sort) {
                      Sort sort = (Sort) node;
                      if (sort.fetch != null) {
                          return (double) RexLiteral.intValue(sort.fetch);
                      }
                  }
              }
              
              return Double.POSITIVE_INFINITY;
          }
      }
      

        Attachments

        1. CALCITE-1018.01.patch
          1 kB
          Pengcheng Xiong
        2. CALCITE-1018.repro.diff
          5 kB
          Maryann Xue

          Issue Links

            Activity

              People

              • Assignee:
                maryannxue Maryann Xue
                Reporter:
                maryannxue Maryann Xue
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: