Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1803

Push Project that follows Aggregate down to Druid

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.11.0
    • Fix Version/s: 1.14.0
    • Component/s: druid
    • Labels:
      None

      Description

      Druid post aggregations are not supported when parsing SQL queries. By implementing post aggregations, we can offload some computation to the druid cluster rather than aggregate on the client side.

      Example usage:
      SELECT SUM("column1") - SUM("column2") FROM "table";
      This query will be parsed into two separate Druid aggregations according to current rules. Then the results will be subtracted in Calcite. By using the postAggregations field in the druid query, the subtraction could be done in Druid cluster. Although the previous example is simple, the difference will be obvious when the number of result rows are large. (Multiple rows result will happen when group by is used).
      Questions:
      After I push Post aggregation into Druid query, what should I change on the project relational correlation? In the case of the example above, the BindableProject will have the expression to representation the subtraction. If I push the post aggregation into druid query, the expression of subtraction should be replaced by the representation of the post aggregations result. For now, the project expression seems can only point to the aggregations results. Since post aggregations have to point to aggregations results too, it could not be placed in the parallel level as aggregation. Where should I put post aggregations?

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                axeisghost Junxian Wu
                Reporter:
                axeisghost Junxian Wu
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 336h
                  336h
                  Remaining:
                  Remaining Estimate - 336h
                  336h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified