Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1812

Provide RelMetadataQuery from planner to rules and invalidate in transformTo

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.13.0
    • Component/s: core
    • Labels:

      Description

      Currently rules create a needed RelMetadataQuery when matched (ie. RuleXxx.onMatch. Since rules are called often, on complex query this triggers a multiplication RMQ instances used. But each new instance carries a heavy price on performance, because is the RMQ instance that hosts the cache used to memoize various metadata lookups (like getEstimatedRows, getPulledUpPredicates, uniqueness and so on). From HIVE-16757 investigation, the performance impact can be 10x.

      The proposal is to have a RelMetadataQuery in the planner and pass it to the rules as a RelOptRuleCall property. Rules onMatch can make use of this instance. The rule call transformTo can handle invalidating the RMQ instance.

      If code knowingly invalidates metadata properties associated with a node by means other than transformTo, it should somehow invalidate the current RMQ.

      The RMQ invalidation can be made smarter, but my plan right now is to do the dumb thing and just toss it away. Will revisit this later, depending on measured impact.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                rusanu Remus Rusanu
                Reporter:
                rusanu Remus Rusanu
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: