[ARROW-15758] [C++] Explore upgrading to mimalloc V2

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 7.0.0
• Component/s: C++
• External issue URL: https://github.com/apache/arrow/issues/31203
• Assignee: Unassigned
• Reporter: Will Jones
• Created: 23/Feb/22 00:08
• Updated: 11/Jan/23 11:39
• Resolved: 03/Jun/22 09:21

Description

ARROW-15730 found that mimalloc wasn't releasing memory as expected. These memory allocators tend to hold onto memory longer than users expect, which can be confusing. But also there appears to be a bug where it also doesn't reuse memory. Both of these are addressed in v2.0.X (beta) of the library: the allocation is more aggressive in returning memory and the bug seems to not exist.

According to one of the maintainers, the main reason 2.0.X hasn't been declared stable is that some use cases have reported performance regressions. We could create a branch of Arrow using mimalloc v2 and run conbench benchmarks to see comparisons. If it's faster, we may consider moving forward; if not, we could provide feedback to the mimalloc maintainers which may help development along.
 

Issue Links

• is superceded by: ARROW-16731 [C++] Bump version of bundled mimalloc