Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-1096

[C++] Memory mapping file over 4GB fails on Windows

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.4.0
    • Fix Version/s: 0.5.0
    • Component/s: C++, Python
    • Labels:
      None

      Description

      This succeeds on a 3.4 GB file, but fails on a 6GB file. I suspect the failure point is exactly 4GB but will investigate

      In [9]: mm = pyarrow.memory_map('df_big.feather')
      ---------------------------------------------------------------------------
      ArrowIOError                              Traceback (most recent call last)
      <ipython-input-9-3160805e299d> in <module>()
      ----> 1 mm = pyarrow.memory_map('df_big.feather')
      
      pyarrow\io.pxi in pyarrow.lib.memory_map (C:\bld\pyarrow_1495576136174\work\arrow-a8f8ba0cbcf5f596f042e90b7a208e7a0c3925b7\python\build\temp.win-amd64-3.6\Release\lib.cxx:39905)()
      
      pyarrow\io.pxi in pyarrow.lib.MemoryMappedFile.open (C:\bld\pyarrow_1495576136174\work\arrow-a8f8ba0cbcf5f596f042e90b7a208e7a0c3925b7\python\build\temp.win-amd64-3.6\Release\lib.cxx:39709)()
      
      pyarrow\error.pxi in pyarrow.lib.check_status (C:\bld\pyarrow_1495576136174\work\arrow-a8f8ba0cbcf5f596f042e90b7a208e7a0c3925b7\python\build\temp.win-amd64-3.6\Release\lib.cxx:6932)()
      
      ArrowIOError: IOError: Memory mapping file failed, errno: 5
      

        Attachments

          Activity

            People

            • Assignee:
              Max Risuhin Max Risuhin
              Reporter:
              wesm Wes McKinney
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: