Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-18186

Migrate HiveUDAFFunction to TypedImperativeAggregate for partial aggregation support

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.6.2, 2.0.1
    • Fix Version/s: 2.2.0
    • Component/s: SQL
    • Labels:
      None
    • Target Version/s:

      Description

      Currently, Hive UDAFs in Spark SQL don't support partial aggregation. Any query involving any Hive UDAFs has to fall back to SortAggregateExec without partial aggregation.

      This issue can be fixed by migrating HiveUDAFFunction to TypedImperativeAggregate, which already provides partial aggregation support for aggregate functions that may use arbitrary Java objects as aggregation states.

        Attachments

          Activity

            People

            • Assignee:
              lian cheng Cheng Lian
              Reporter:
              lian cheng Cheng Lian
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: