[HDFS-6750] The DataNode should use its shared memory segment to mark short-circuit replicas that have been unlinked as stale

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 2.6.0
• Component/s: datanode, hdfs-client
• Target Version/s: 2.6.0
• Assignee: Colin McCabe
• Reporter: Colin McCabe
• Created: 25/Jul/14 00:30
• Updated: 12/May/15 19:24
• Resolved: 25/Jul/14 23:57

Description

The DataNode should mark short-circuit replicas that have been unlinked as stale. This would prevent replicas that had been deleted from lingering in the DFSClient cache. (At least for DFSClients that use shared memory; those without shared memory will still have to use the timeout method.)

Note that when a replica is stale, any ongoing reads or mmaps can still complete. But stale replicas will be removed from the DFSClient cache once they're no longer in use.

Issue Links

• is duplicated by: HDFS-7900 ShortCircuitCache.replicaInfoMap keeps too many deleted file descriptor
• relates to: HDFS-6604 The short-circuit cache doesn't correctly time out replicas that haven't been used in a while