[HADOOP-11935] Provide optional native implementation of stat syscall.

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: fs, native
• Assignee: Unassigned
• Reporter: Chris Nauroth
• Created: 07/May/15 16:27
• Updated: 28/Apr/19 12:16

Description

Currently, RawLocalFileSystem.DeprecatedRawLocalFileStatus#loadPermissionInfo is implemented as forking an ls command and parsing the output. This was observed to be a bottleneck in YARN-3491. This issue proposes an optional native implementation of a stat syscall through JNI. We would maintain the existing code as a fallback for systems where the native code is not available.

Issue Links

• duplicates: HADOOP-14600 LocatedFileStatus constructor forces RawLocalFS to exec a process to get the permissions
• is depended upon by: YARN-3549 use JNI-based FileStatus implementation from io.nativeio.NativeIO.POSIX#getFstat instead of shell-based implementation from RawLocalFileSystem in checkLocalDir.
• is duplicated by: YARN-3549 use JNI-based FileStatus implementation from io.nativeio.NativeIO.POSIX#getFstat instead of shell-based implementation from RawLocalFileSystem in checkLocalDir.
• is related to: HADOOP-12603 TestSymlinkLocalFSFileContext#testSetTimesSymlinkToDir occasionally fail
• links to: GitHub Pull Request #81