Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4366 Aggregation Improvement
  3. SPARK-9830

Remove AggregateExpression1 and Aggregate Operator used to evaluate AggregateExpression1s

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • None
    • 1.6.0
    • SQL
    • None

    Description

      In 1.6.0, we should remove the old aggregation code path (used to evaluate AggregateExpression1). While removing this code path, there are several code clean up we need to do,
      1. Remove all of our hacks from ResolveFunctions.
      2. Remove all of the conversion logics from {org.apache.spark.sql.catalyst.expressions.aggregate.utils}}.
      3. Remove newAggregation field from logical.Aggregate.
      4. Remove the query planning rule for old aggregate path.

      Attachments

        Issue Links

          is blocked by

          Sub-task - The sub-task of the issue SPARK-9241 Supporting multiple DISTINCT columns

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Sub-task - The sub-task of the issue SPARK-11017 Support ImperativeAggregates in TungstenAggregate

          • Major - Major loss of function.
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-11645 Remove OpenHashSet for the old aggregate.

          • Major - Major loss of function.
          • Resolved
          links to

          Pull request #9556 [Github] Pull Request #9556 (yhuai)

          Pull request #9607 [Github] Pull Request #9607 (yhuai)

          Activity

            People

              yhuai Yin Huai
              yhuai Yin Huai
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: