[ARROW-3936] Add _O_NOINHERIT to the file open flags on Windows

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.12.0
• Component/s: C++
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/20544
• Assignee: Unassigned
• Reporter: Philip Felton
• Created: 04/Dec/18 14:40
• Updated: 11/Jan/23 07:30
• Resolved: 05/Dec/18 10:59

Description

Unlike Linux, Windows doesn't let you delete files that are currently opened by another process. So if you create a child process while a Parquet file is open, with the current code the file handle is inherited to the child process, and the parent process can't then delete the file after closing it without the child process terminating first.

By default, Win32 file handles are not inheritable (likely because of the aforementioned problems). Except for _wsopen_s, which tries to maintain POSIX compatibility.

This is a serious problem for us.

We would argue that specifying _O_NOINHERIT by default in the _MSC_VER path is a sensible approach and would likely be the correct behaviour as it matches the main Win32 API.

However, it could be that some developers rely on the current inheritable behaviour. In which case, the Arrow public API should take a boolean argument on whether the created file descriptor should be inheritable. But this would break API backward compatibility (unless a new overloaded method is introduced).

Is forking and inheriting Arrow internal file descriptor something that Arrow actually means to support?

See https://github.com/apache/arrow/pull/3085. What do we think of the proposed fix?

Issue Links

• links to: GitHub Pull Request #3085