Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-2223

ProjectMergeRule is infinitely matched when is applied after ProjectReduceExpressionsRule

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Critical
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      For queries like this:

      select t1.f from (select cast(f as int) f, f from (select cast(f as int) f from (values('1')) t(f))) as t1
      

      OOM is thrown when ProjectMergeRule is applied before applying ProjectReduceExpressionsRule in VolcanoPlanner.
      A simple test to reproduce this issue (in RelOptRulesTest):

        @Test public void testOomProjectMergeRule() {
          RelBuilder relBuilder = RelBuilder.create(RelBuilderTest.config().build());
          RelNode relNode = relBuilder
              .values(new String[]{"f"}, "1")
              .project(
                  relBuilder.alias(
                      relBuilder.cast(relBuilder.field(0), SqlTypeName.INTEGER),
                      "f"))
              .project(
                  relBuilder.alias(
                      relBuilder.cast(relBuilder.field(0), SqlTypeName.INTEGER),
                      "f0"),
                  relBuilder.alias(relBuilder.field(0), "f"))
              .project(
                  relBuilder.alias(relBuilder.field(0), "f"))
              .build();
      
          RelOptPlanner planner = relNode.getCluster().getPlanner();
          RuleSet ruleSet =
              RuleSets.ofList(
                  ReduceExpressionsRule.PROJECT_INSTANCE,
                  new ProjectMergeRuleWithLongerName(),
                  EnumerableRules.ENUMERABLE_PROJECT_RULE,
                  EnumerableRules.ENUMERABLE_VALUES_RULE);
          Program program = Programs.of(ruleSet);
      
          RelTraitSet toTraits =
              relNode.getCluster().traitSet()
                  .replace(0, EnumerableConvention.INSTANCE);
      
          RelNode output = program.run(planner, relNode, toTraits,
              ImmutableList.<RelOptMaterialization>of(), ImmutableList.<RelOptLattice>of());
      
          // check for output
        }
      
        /**
         * ProjectMergeRule inheritor which has
         * class name greater than ProjectReduceExpressionsRule class name (String.compareTo()).
         *
         * It is needed for RuleQueue.popMatch() method
         * to apply this rule before ProjectReduceExpressionsRule.
         */
        private static class ProjectMergeRuleWithLongerName extends ProjectMergeRule {
          public ProjectMergeRuleWithLongerName() {
            super(true, RelFactories.LOGICAL_BUILDER);
          }
        }
      

      Attachments

        1. provenance_contents.png
          175 kB
          Vladimir Sitnikov
        2. heap_overview.png
          530 kB
          Vladimir Sitnikov
        3. TestLimitWithExchanges_testPushLimitPastUnionExchange.png
          428 kB
          Vladimir Sitnikov

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            volodymyr Vova Vysotskyi

            Dates

              Created:
              Updated:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 10m
                10m

                Slack

                  Issue deployment