Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-12913

Reimplement stat functions as declarative function

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: SQL
    • Labels:
      None

      Description

      As benchmarked and discussed here: https://github.com/apache/spark/pull/10786/files#r50038294.

      Benefits from codegen, the declarative aggregate function could be much faster than imperative one, we should re-implement all the builtin aggregate functions as declarative one.

      For skewness and kurtosis, we need to benchmark it to make sure that the declarative one is actually faster than imperative one.

        Attachments

          Activity

            People

            • Assignee:
              davies Davies Liu
              Reporter:
              davies Davies Liu
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: