[ARROW-8898] [C++] Determine desirable maximum length for ExecBatch in pipelined and parallel execution of kernels

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: C++
• External issue URL: https://github.com/apache/arrow/issues/25032
• Assignee: Unassigned
• Reporter: Wes McKinney
• Created: 22/May/20 21:46
• Updated: 11/Jan/23 08:03

Description

Maximum lengths like 16K or 64K seem to be popular, but we should write our own benchmarks so that we can justify the choice of default chunksize

Issue Links

• is a child of: ARROW-8894 [C++] C++ array kernels framework and execution buildout (umbrella issue)
• relates to: ARROW-16755 [C++] Improve array expression and kernel evaluation performance on small inputs