Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-158

Optiq fails when call Planner.transform() multiple times, each with different ruleset.

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      I'm trying to call Planner.transform(int ruleSetIndex,RelTraitSet requiredOutputTraits, RelNode rel) multiple times, and Optiq fails during the 2nd call of transform, complaining AssertionError "Used match list for phase PRE_PROCESS_MDR after phase complete" by org.eigenbase.relopt.volcano.RuleQueue.popMatch.

      The reason for this error is because during the first call of transform(), in VolcanoPlanner.findBestExp(), it will remove the PhaseMatchList for each phase in rule queue, once that phase is completed:
      ruleQueue.phaseCompleted(phase);

      In the following call of transform(), even there are new matched rules, those rules will not be able to be added to rule queue, since there is no PhaseMatchList for the phase in the rule queue.

      To address this issue, is it possible that optiq changes as following:
      1) In RuleQueue.phaseCompleted() call, we just clean up the PhaseMatchList, in stead of remove PhaseMatchList from rule queue.
      2) Move the logic of ruleQueue.phaseCompleted(phase) to Planner.clearRules()? That is, clearRules() will clear both the ruleset as well as rule queue ? Or is there any particular reason that the rule queue has to be cleaned up within findBestExp() call ?

      Thank you,

      ---------------- Imported from GitHub ----------------
      Url: https://github.com/julianhyde/optiq/issues/158
      Created by: jinfengni
      Labels:
      Created at: Tue Feb 25 20:27:49 CET 2014
      State: closed

        Activity

        Hide
        github-import GitHub Import added a comment -

        [Date: Wed Feb 26 02:54:02 CET 2014, Author: jinfengni]

        Hi Julian,

        Thanks for your quick fix. The fix makes one step ahead. However, I still hit some problem when I call PlannerImpl.transform() twice. Basically, what I did is following:

        RelNode node1 = PlannerImpl.transform(first_rule_set, inputRelNode);
        RelNode node2 = PlannerImpl.transform(second_rule_set, node1);

        The 2nd call will take the result (node1) as the input, and would like to apply a different set of rules against the tree with root = node1. However, since the planner still keeps the internal status of the previous call (mapRel2Subset, relImportance, etc), the 2nd will recognize the equivalent RelSubset during registerImpl() call, and then hit some error.

        To address this problem, my temp fix is to add a clear() method to RelOptPlanner, which will clear the internal status of the planner. PlannerImpl.transform() will call this RelOptPlanner.clear() to make sure the status from the prior transform() call is cleaned up.

        I tried this approach in Drill code base, and it works in the way that I expect ( the second rule sets are fired properly ).

        I put my temp code fix here https://github.com/jinfengni/optiq/tree/PlannerClear

        Show
        github-import GitHub Import added a comment - [Date: Wed Feb 26 02:54:02 CET 2014, Author: jinfengni ] Hi Julian, Thanks for your quick fix. The fix makes one step ahead. However, I still hit some problem when I call PlannerImpl.transform() twice. Basically, what I did is following: RelNode node1 = PlannerImpl.transform(first_rule_set, inputRelNode); RelNode node2 = PlannerImpl.transform(second_rule_set, node1); The 2nd call will take the result (node1) as the input, and would like to apply a different set of rules against the tree with root = node1. However, since the planner still keeps the internal status of the previous call (mapRel2Subset, relImportance, etc), the 2nd will recognize the equivalent RelSubset during registerImpl() call, and then hit some error. To address this problem, my temp fix is to add a clear() method to RelOptPlanner, which will clear the internal status of the planner. PlannerImpl.transform() will call this RelOptPlanner.clear() to make sure the status from the prior transform() call is cleaned up. I tried this approach in Drill code base, and it works in the way that I expect ( the second rule sets are fired properly ). I put my temp code fix here https://github.com/jinfengni/optiq/tree/PlannerClear

          People

          • Assignee:
            Unassigned
            Reporter:
            github-import GitHub Import
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development