[HDFS-3767] Finer grained locking in DN

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.0-alpha1
• Component/s: performance
• Assignee: Yong Zhang
• Reporter: Todd Lipcon
• Created: 06/Aug/12 19:57
• Updated: 12/May/16 18:13

Description

In testing a high-write-throughput workload, I see the DN maintain good performance most of the time, except that occasionally one thread will block for a few seconds in finalizeReplica. It does so holding the FSDatasetImpl lock, which causes all other writer threads to block behind it. HDFS-1148 (making it a rw lock) would help here, but a bigger help would be to go do finer-grained locking (eg per block or per-subdir).

Issue Links

• is related to: HDFS-1148 Convert FSDataset to ReadWriteLock