[ARROW-1096] [C++] Memory mapping file over 4GB fails on Windows

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.4.0
• Fix Version/s: 0.5.0
• Component/s: C++, Python
• External issue URL: https://github.com/apache/arrow/issues/16683
• Assignee: Max Risuhin
• Reporter: Wes McKinney
• Created: 07/Jun/17 03:49
• Updated: 11/Jan/23 07:12
• Resolved: 16/Jun/17 01:05

Description

This succeeds on a 3.4 GB file, but fails on a 6GB file. I suspect the failure point is exactly 4GB but will investigate

In [9]: mm = pyarrow.memory_map('df_big.feather')
---------------------------------------------------------------------------
ArrowIOError                              Traceback (most recent call last)
<ipython-input-9-3160805e299d> in <module>()
----> 1 mm = pyarrow.memory_map('df_big.feather')

pyarrow\io.pxi in pyarrow.lib.memory_map (C:\bld\pyarrow_1495576136174\work\arrow-a8f8ba0cbcf5f596f042e90b7a208e7a0c3925b7\python\build\temp.win-amd64-3.6\Release\lib.cxx:39905)()

pyarrow\io.pxi in pyarrow.lib.MemoryMappedFile.open (C:\bld\pyarrow_1495576136174\work\arrow-a8f8ba0cbcf5f596f042e90b7a208e7a0c3925b7\python\build\temp.win-amd64-3.6\Release\lib.cxx:39709)()

pyarrow\error.pxi in pyarrow.lib.check_status (C:\bld\pyarrow_1495576136174\work\arrow-a8f8ba0cbcf5f596f042e90b7a208e7a0c3925b7\python\build\temp.win-amd64-3.6\Release\lib.cxx:6932)()

ArrowIOError: IOError: Memory mapping file failed, errno: 5