[HADOOP-752] Possible locking issues in HDFS Namenode

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.10.0
• Assignee: Dhruba Borthakur
• Reporter: Dhruba Borthakur
• Created: 28/Nov/06 05:51
• Updated: 08/Jul/09 16:42
• Resolved: 07/Dec/06 18:46

Description

I have been investigating the cause of random Namenode memory corruptions/memory overflows, etc. Please comment.

1. The functions datanodeReport() and DFSNodesStatus() do not acquire the global lock.
This can race with another thread invoking registerDatanode(). registerDatanode()
can remove a datanode (thru wipeDatanode()) while the datanodeReport thread is
traversing the list of datanodes. This can cause exceptions to occur.

2. The blocksMap is protected by the global lock. The setReplication() call does not acquire
the global lock when it calls proccessOverReplicatedBlock(). This can cause corruption in blockMap.

Issue Links

• is related to: HADOOP-751 Namenode constantly using up 100% CPU