[HDFS-15382] Split one FsDatasetImpl lock to volume grain locks.

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: datanode
• Labels: pull-request-available
• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed

Release Note

Throughput is one of the core performance evaluation for DataNode instance. However it does not reach the best performance especially for Federation deploy all the time although there are different improvement, because of the global coarse-grain lock. These series issues (include HDFS-16534, HDFS-16511, HDFS-15382 and HDFS-16429.) try to split the global coarse-grain lock to fine-grain lock which is double level lock for blockpool and volume, to improve the throughput and avoid lock impacts between blockpools and volumes.

• Assignee: Mingxiang Li
• Reporter: Mingxiang Li
• Created: 02/Jun/20 04:45
• Updated: 11/Feb/24 10:28
• Resolved: 12/Mar/22 10:49

Description

In HDFS-15180 we split lock to blockpool grain size.But when one volume is in heavy load and will block other request which in same blockpool but different volume.So we split lock to two leval to avoid this happend.And to improve datanode performance.

Issue Links

• Dependent: HDFS-15180 DataNode FsDatasetImpl Fine-Grained Locking via BlockPool.
• is duplicated by: HDFS-17203 ReplicaMap#map should better use ConcurrentHashMap rather than HashMap
• is related to: HDFS-16783 Remove the redundant lock in deepCopyReplica and getFinalizedBlocks
• is related to: HDFS-16785 Avoid to hold write lock to improve performance when add volume.
• is related to: HDFS-16787 Remove the redundant lock in DataSetLockManager#removeLock.
• relates to: HDFS-16855 Remove the redundant write lock in addBlockPool
• links to: GitHub Pull Request #3941
• links to: GitHub Pull Request #4141
• links to: https://github.com/apache/hadoop/pull/4141

Sub-Tasks

• Add DataSetLockManager to manage fine-grain locks for FsDataSetImpl: Resolved
• DataNode FsDatasetImpl Fine-Grained Locking via BlockPool.: Resolved
• Improve lock type for ReplicaMap under fine-grain lock mode.: Resolved
• Avoid to hold write lock to improve performance when add volume.: Resolved
• Split datanode block pool locks to volume grain.: Resolved
• Fix DataNode FsDatasetImpl lock issue without GS checks.: Resolved
• Fix deadlock of fine-grain lock for FsDatastImpl of DataNode.: Resolved
• Changing pool-level lock to volume-level lock for invalidation of blocks: Open
• Remove the redundant lock in deepCopyReplica and getFinalizedBlocks: Resolved
• Remove the redundant lock in DataSetLockManager#removeLock.: Resolved
• Move FsVolumeList#getVolumes() invocation out of DataSetLock: Open
• FsVolumeList#getNextVolume can be moved out of DataSetLock: Open