Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-752

Possible locking issues in HDFS Namenode

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0
    • Component/s: None
    • Labels:
      None

      Description

      I have been investigating the cause of random Namenode memory corruptions/memory overflows, etc. Please comment.

      1. The functions datanodeReport() and DFSNodesStatus() do not acquire the global lock.
      This can race with another thread invoking registerDatanode(). registerDatanode()
      can remove a datanode (thru wipeDatanode()) while the datanodeReport thread is
      traversing the list of datanodes. This can cause exceptions to occur.

      2. The blocksMap is protected by the global lock. The setReplication() call does not acquire
      the global lock when it calls proccessOverReplicatedBlock(). This can cause corruption in blockMap.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                dhruba dhruba borthakur
                Reporter:
                dhruba dhruba borthakur
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: