[ARROW-12044] [C++][Compute] Add support for imperfect grouping for use in radix partitioning

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: C++
• Labels: query-engine
• External issue URL: https://github.com/apache/arrow/issues/27873
• Assignee: Unassigned
• Reporter: Ben Kietzman
• Created: 22/Mar/21 17:05
• Updated: 11/Jan/23 08:23

Description

ARROW-11591 adds Grouper for identifying groups based on multiple key columns.

For a large number of groups, it is beneficial to do a first pass partitioning on the key columns so that each worker thread only handles a subset of the query's groups. This is usually accomplished by computing only the hashes of the keys (not full group identity) and pushing slices of the input batches to workers based on those.

This would probably make sense as a member function of Grouper, maybe Grouper::ConsumeImperfect

Issue Links

• is a child of: ARROW-12633 [C++] Query engine umbrella issue