Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-3915

Add rule listener to report rule attempts and time at DEBUG log level

Rank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersConvert to sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 1.23.0
    • None

    Description

      We can add a rule listener to the VolcanoPlanner for recording rule attempts and the accumulative time for the rule firing. This would be very helpful to debug planner performance issue (such as CALCITE-2970). The listener will only be added under DEBUG log level.

      It will produce a rule attempt summary like below -

      2020-04-10 18:03:24,621 [main] DEBUG - 
      Rules                                                                   Attempts           Time (us)
      ProjectMergeRule:force_mode                                                1,203           1,124,242
      EnumerableProjectRule(in:NONE,out:ENUMERABLE)                                129             226,475
      AggregatePullUpConstantsRule                                                  84               1,949
      AggregateProjectMergeRule                                                     81             128,575
      ProjectFilterTransposeRule                                                    76               8,734
      JoinPushExpressionsRule                                                       71              10,314
      FilterJoinRule:FilterJoinRule:no-filter                                       71               1,306
      JoinPushThroughJoinRule:right                                                 47                 676
      JoinPushThroughJoinRule:left                                                  45             175,696
      FilterJoinRule:FilterJoinRule:filter                                          41              89,381
      ReduceExpressionsRule(Filter)                                                 24              38,128
      EnumerableFilterRule(in:NONE,out:ENUMERABLE)                                  24              52,457
      EnumerableJoinRule(in:NONE,out:ENUMERABLE)                                    24              71,752
      EnumerableMergeJoinRule(in:NONE,out:ENUMERABLE)                               24              47,987
      FilterProjectTransposeRule                                                    22              55,177
      JoinCommuteRule                                                               20              70,240
      EnumerableAggregateRule(in:NONE,out:ENUMERABLE)                               10              26,522
      AggregateExpandDistinctAggregatesRule                                         10                  37
      EnumerableTableScanRule(in:NONE,out:ENUMERABLE)                                1                 527
      EnumerableInterpreterRule(in:BINDABLE,out:ENUMERABLE)                          1                 764
      BindableTableScanRule                                                          1               1,710
      ExpandConversionRule                                                           1                 175
      

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            xndai Xiening Dai
            xndai Xiening Dai
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 7h 20m
                7h 20m

                Slack

                  Issue deployment