[CALCITE-2223] ProjectMergeRule is infinitely matched when is applied after ProjectReduceExpressionsRule

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Vova Vysotskyi
• Created: 25/Mar/18 16:45
• Updated: 07/Dec/22 07:44
• Resolved: 20/Jul/22 09:23

Description

For queries like this:

select t1.f from (select cast(f as int) f, f from (select cast(f as int) f from (values('1')) t(f))) as t1

OOM is thrown when ProjectMergeRule is applied before applying ProjectReduceExpressionsRule in VolcanoPlanner.
A simple test to reproduce this issue (in RelOptRulesTest):

  @Test public void testOomProjectMergeRule() {
    RelBuilder relBuilder = RelBuilder.create(RelBuilderTest.config().build());
    RelNode relNode = relBuilder
        .values(new String[]{"f"}, "1")
        .project(
            relBuilder.alias(
                relBuilder.cast(relBuilder.field(0), SqlTypeName.INTEGER),
                "f"))
        .project(
            relBuilder.alias(
                relBuilder.cast(relBuilder.field(0), SqlTypeName.INTEGER),
                "f0"),
            relBuilder.alias(relBuilder.field(0), "f"))
        .project(
            relBuilder.alias(relBuilder.field(0), "f"))
        .build();

    RelOptPlanner planner = relNode.getCluster().getPlanner();
    RuleSet ruleSet =
        RuleSets.ofList(
            ReduceExpressionsRule.PROJECT_INSTANCE,
            new ProjectMergeRuleWithLongerName(),
            EnumerableRules.ENUMERABLE_PROJECT_RULE,
            EnumerableRules.ENUMERABLE_VALUES_RULE);
    Program program = Programs.of(ruleSet);

    RelTraitSet toTraits =
        relNode.getCluster().traitSet()
            .replace(0, EnumerableConvention.INSTANCE);

    RelNode output = program.run(planner, relNode, toTraits,
        ImmutableList.<RelOptMaterialization>of(), ImmutableList.<RelOptLattice>of());

    // check for output
  }

  /**
   * ProjectMergeRule inheritor which has
   * class name greater than ProjectReduceExpressionsRule class name (String.compareTo()).
   *
   * It is needed for RuleQueue.popMatch() method
   * to apply this rule before ProjectReduceExpressionsRule.
   */
  private static class ProjectMergeRuleWithLongerName extends ProjectMergeRule {
    public ProjectMergeRuleWithLongerName() {
      super(true, RelFactories.LOGICAL_BUILDER);
    }
  }

Issue Links

• Blocked: CALCITE-3041 Query runs endless when setImportance to 0
• is depended upon by: DRILL-6371 Use FilterSetOpTransposeRule, DrillProjectSetOpTransposeRule in main logical stage
• is related to: CALCITE-1271 Enable FilterSetOpTransposeRule, AggregateUnionTransposeRule in default rule set
• is related to: CALCITE-3505 Infinite matching of FilterProjectTransposeRule causes stackoverflow
• is related to: DRILL-3855 Enable FilterSetOpTransposeRule, DrillProjectSetOpTransposeRule
• relates to: DRILL-6212 A simple join is recursing too deep in planning and eventually throwing stack overflow.
• links to: GitHub Pull Request #855
• links to: PR 855