Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-5177

Query loses hint after decorrelation

    XMLWordPrintableJSON

Details

    Description

      This seems to be a regression caused by CALCITE-5107, which enlarged the list of hintable RelNodes by making Filter, SetOp, Sort, Window, Values hintable.
      However, it seems that this patch "missed" some Calcite preprocessing elements that can change a plan, and therefore require a manual / ad-hoc treatment of hints to avoid losing them in their processing, e.g. RelDecorrelator.

      In my specific example, let's have this query:

      SELECT /*+ MY_HINT_FOR_JOIN */ c.c_custkey, , c.c_name FROM customer c
      WHERE NOT EXISTS (
        SELECT 1 FROM orders o WHERE o.o_custkey = c.c_custkey AND o.o_orderstatus <> 'abc'
      ) ORDER BY c.c_custkey
      

      Before CALCITE-5107, when the query was parsed, a logical plan was created, my hint was attached to a LogicalProject. The plan contained a LogicalCorrelate (to implement the NOT EXISTS). The plan was then decorrelated with RelDecorrelator, and as a result, we obtained a new plan with a LogicalJoin (instead of correlate), where the hint had been propagated from the projection until the join. Everything is fine.

      After CALCITE-5107, when the query was parsed, now we obtain the same logical plan, but now the hint is attached to the LogicalSort (not to the LogicalProject). When the decorrelator is executed, the plan is transformed (to have LogicalJoin), but the hint is lost, it is not in the new plan's sort (or project, or join, it's nowhere).
      Running the debugger, it seems the problem is inside RelDecorrelator#decorrelateQuery:

      ...
        if (!decorrelator.cm.mapCorToCorRel.isEmpty()) {
          newRootRel = decorrelator.decorrelate(newRootRel); // <-- HINT LOST HERE!
        }
        // NOTHING GETS PROPAGATED BECAUSE THE HINT WAS LOST!
        newRootRel = RelOptUtil.propagateRelHints(newRootRel, true);
        
        return newRootRel;
      

      The root cause seems to be that, inside RelDecorrelator's code, there are only a few places where hints are copied from the "old" RelNode to the newly created RelNode: 3 calls to RelOptUtil#copyRelHints (here, here, and here), only for projections and aggregates; + RelBuilder#hints to copy them for joins.
      To sum up, it seems RelDecorrelator only cared to copy hints for the "traditional hintables", so probably something like that is missing for sorts (and other newly hintable RelNodes added by CALCITE-5107).

      Apart from RelDecorrelator, other classes that might suffer from a similar problem are:

      Attachments

        Issue Links

          Activity

            People

              rubenql Ruben Q L
              rubenql Ruben Q L
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 4h
                  4h