Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-13611

import Aggregator doesn't work in Spark Shell

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Incomplete
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: SQL
    • Labels:

      Description

      scala> import org.apache.spark.sql.expressions.Aggregator
      import org.apache.spark.sql.expressions.Aggregator
      
      scala> class SumOf[I, N : Numeric](f: I => N) extends Aggregator[I, N, N] with Serializable {
           |   val numeric = implicitly[Numeric[N]]
           |   override def zero: N = numeric.zero
           |   override def reduce(b: N, a: I): N = numeric.plus(b, f(a))
           |   override def merge(b1: N,b2: N): N = numeric.plus(b1, b2)
           |   override def finish(reduction: N): N = reduction
           | }
      <console>:10: error: not found: type Aggregator
             class SumOf[I, N : Numeric](f: I => N) extends Aggregator[I, N, N] with Serializable {
                                                            ^
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                cloud_fan Wenchen Fan
              • Votes:
                1 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: