[SPARK-18728] Consider using Algebird's Aggregator instead of org.apache.spark.sql.expressions.Aggregator

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Incomplete
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Alex Levenson
• Created: 05/Dec/16 23:03
• Updated: 21/May/19 04:12
• Resolved: 21/May/19 04:12

Description

Mansur (https://twitter.com/mansur_ashraf) pointed out this comment in spark's Aggregator here:
"Based loosely on Aggregator from Algebird: https://github.com/twitter/algebird"
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/expressions/Aggregator.scala#L46

Which got a few of us wondering, given that this API is still experimental, would you consider using algebird's Aggregator API directly instead?

The algebird API is not coupled with any implementation details, and shouldn't have any extra dependencies.

Are there any blockers to doing that?

Thanks!
Alex