[ARROW-17115] [C++] HashJoin fails if it encounters a batch with more than 32Ki rows

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 9.0.0
• Component/s: C++
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/32419
• Assignee: Weston Pace
• Reporter: Weston Pace
• Created: 18/Jul/22 23:20
• Updated: 11/Jan/23 11:48
• Resolved: 22/Jul/22 21:21

Description

The new swiss join assumes that batches are being broken according to the morsel/batch model and it assumes those batches have, at most, 32Ki rows (signed 16-bit indices are used in various places).

However, we are not currently slicing all of our inputs to batches this small. This is causing conbench to fail and would likely be a problem with any large inputs.

We should fix this by slicing batches in the engine to the appropriate maximum size.

Issue Links

• links to: GitHub Pull Request #13679