[HADOOP-15486] Make NetworkTopology#netLock fair

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.10.0, 3.2.0, 3.1.1, 2.9.2, 3.0.3, 2.7.7, 2.8.5
• Component/s: net
• Hadoop Flags: Reviewed
• Assignee: Nandakumar
• Reporter: Nandakumar
• Created: 21/May/18 19:28
• Updated: 23/Jul/18 17:14
• Resolved: 23/May/18 17:35

Description

Whenever a datanode is restarted, the registration call after the restart received by NameNode lands in NetworkTopology#add via DatanodeManager#registerDatanode requires write lock on NetworkTopology#netLock.

This registration thread is getting starved by flood of FSNamesystem.getAdditionalDatanode calls, which are triggered by clients those who were writing to the restarted datanode.

The registration call which is waiting for write lock on NetworkTopology#netLock is holding write lock on FSNamesystem#fsLock, causing all the other RPC calls which require the lock on FSNamesystem#fsLock wait.
We can make NetworkTopology#netLock lock fair so that the registration thread will not starve.

Issue Links

• is depended upon by: HADOOP-15509 Release Hadoop 2.7.7