[ARROW-4124] [C++] Abstract aggregation kernel API

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.13.0
• Component/s: C++
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/20713
• Assignee: Francois Saint-Jacques
• Reporter: Wes McKinney
• Created: 27/Dec/18 17:53
• Updated: 11/Jan/23 07:31
• Resolved: 10/Feb/19 01:05

Description

Related to the particular details of implementing various aggregation types, we should first put a bit of energy into the abstract API for aggregating data in a multi-threaded setting

Aggregators must support both hash/group (e.g. "group by" in SQL or data frame libraries) modes and non-group modes.

Aggregations ideally should also support filter pushdown. For example:

select $AGG($EXPR)
from $TABLE
where $PREDICATE

Some systems might materialize the post-predicate / filtered version of $EXPR, then aggregate that. pandas does this for example. Vectorized performance can be much improved by filtering inside the aggregation kernel. How the predicate true/false values are handled may depend on the implementation details of the kernel (e.g. SUM or MEAN will be a bit different from PRODUCT)

Issue Links

• relates to: ARROW-3120 [C++] Parallelize execution of ScalarAggregateFunction
• relates to: ARROW-3121 [C++] Mean kernel aggregate
• relates to: ARROW-3123 [C++] Incremental Count, Count Not Null aggregator
• relates to: ARROW-3122 [C++] Incremental Variance, Standard Deviation aggregators
• links to: GitHub Pull Request #3407