[HDFS-5453] Support fine grain locking in FSNamesystem

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.0.0-alpha, 3.0.0-alpha1
• Component/s: namenode
• Assignee: Daryn Sharp
• Reporter: Daryn Sharp
• Created: 04/Nov/13 15:54
• Updated: 19/Mar/18 08:17

Description

The namesystem currently uses a course grain lock to control access. This prevents concurrent writers in different branches of the tree, and prevents readers from accessing branches that writers aren't using.

Features that introduce latency to namesystem operations, such as cold storage of inodes, will need fine grain locking to avoid degrading the entire namesystem's throughput.

Issue Links

• is duplicated by: HDFS-12803 We should not lock FsNamesystem even we operate a sub directory, we should refinement the lock
• is required by: HDFS-5389 A Namenode that keeps only a part of the namespace in memory
• is required by: HDFS-5324 Make Namespace implementation pluggable in the namenode

Sub-Tasks

• FSNamesystem's fsLock should allow custom implementation: Closed
• Add lock context support to FSNamesystemLock: Patch Available
• Update FSNamesystem path operations to use a lock context: Open
• Add iterator support to INodesInPath: Patch Available
• Expose FSNamesystem lock wait time as metrics: Resolved