Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1288

Avoid doing the same join twice if count(distinct) exists

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.9.0
    • None
    • None

    Description

      When the query has one distinct aggregate and one or more non-distinct aggregates, the join instance need not produce the join-based plan. We can generate multi-phase aggregates. Another approach would be to use grouping sets. However, this transformation will be useful when systems don't support grouping sets and instead rely on the join-based plans (see the plan below)

      select emp.empno, count(*), avg(distinct dept.deptno) 
      from sales.emp emp inner join sales.dept dept 
      on emp.deptno = dept.deptno 
      group by emp.empno
      
      LogicalProject(EMPNO=[$0], EXPR$1=[$1], EXPR$2=[$3])
        LogicalJoin(condition=[IS NOT DISTINCT FROM($0, $2)], joinType=[inner])
          LogicalAggregate(group=[{0}], EXPR$1=[COUNT()])
            LogicalProject(EMPNO=[$0], DEPTNO0=[$9])
              LogicalJoin(condition=[=($7, $9)], joinType=[inner])
                LogicalTableScan(table=[[CATALOG, SALES, EMP]])
                LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
          LogicalAggregate(group=[{0}], EXPR$2=[AVG($1)])
            LogicalAggregate(group=[{0, 1}])
              LogicalProject(EMPNO=[$0], DEPTNO0=[$9])
                LogicalJoin(condition=[=($7, $9)], joinType=[inner])
                  LogicalTableScan(table=[[CATALOG, SALES, EMP]])
                  LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
      

      The more efficient form should look like

      select emp.empno, count(*), avg(distinct dept.deptno) 
      from sales.emp emp inner join sales.dept dept 
      on emp.deptno = dept.deptno 
      group by emp.empno
      
      LogicalAggregate(group=[{0}], EXPR$1=[SUM($2)], EXPR$2=[AVG($1)])
        LogicalAggregate(group=[{0, 1}], EXPR$1=[COUNT()])
          LogicalProject(EMPNO=[$0], DEPTNO0=[$9])
            LogicalJoin(condition=[=($7, $9)], joinType=[inner])
              LogicalTableScan(table=[[CATALOG, SALES, EMP]])
              LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
      

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            gparai Gautam Parai
            gparai Gautam Parai
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment