[HDFS-6604] The short-circuit cache doesn't correctly time out replicas that haven't been used in a while

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 2.5.0
• Component/s: hdfs-client

Environment

Centos 6.5 and distribution Hortonworks Data Platform v2.1

• Target Version/s: 2.5.0
• Hadoop Flags: Reviewed
• Assignee: Colin McCabe
• Reporter: Giuseppe Reina
• Created: 27/Jun/14 15:33
• Updated: 15/Aug/14 05:41
• Resolved: 02/Jul/14 19:46

Description

When HDFS shortcircuit is enabled, the file descriptors of the deleted HDFS blocks are kept open until the cache is full. This prevents the operating system to free the space on disk.

More details on the mailing list thread

Issue Links

• is related to: HDFS-6750 The DataNode should use its shared memory segment to mark short-circuit replicas that have been unlinked as stale