[HDFS-6912] SharedFileDescriptorFactory should not allocate sparse files

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.5.0
• Fix Version/s: 2.6.0
• Component/s: caching

Environment

HDFS Data node, with 8 gb tmpfs in /dev/shm

• Target Version/s: 2.6.0
• Assignee: Colin McCabe
• Reporter: Gopal Vijayaraghavan
• Created: 22/Aug/14 02:29
• Updated: 01/Dec/14 03:09
• Resolved: 15/Sep/14 21:52

Description

SharedFileDescriptor factory should not allocate sparse files. Sparse files can lead to a SIGBUS later in the short-circuit reader when we try to read from the sparse file and memory is not available.

Note that if swap is enabled, we can still get a SIGBUS even with a non-sparse file, since the JVM uses MAP_NORESERVE in mmap.

Issue Links

• relates to: HDFS-6581 Write to single replica in memory