Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15486

Make NetworkTopology#netLock fair

    XMLWordPrintableJSON

    Details

    • Hadoop Flags:
      Reviewed

      Description

      Whenever a datanode is restarted, the registration call after the restart received by NameNode lands in NetworkTopology#add via DatanodeManager#registerDatanode requires write lock on NetworkTopology#netLock.

      This registration thread is getting starved by flood of FSNamesystem.getAdditionalDatanode calls, which are triggered by clients those who were writing to the restarted datanode.

      The registration call which is waiting for write lock on NetworkTopology#netLock is holding write lock on FSNamesystem#fsLock, causing all the other RPC calls which require the lock on FSNamesystem#fsLock wait.
      We can make NetworkTopology#netLock lock fair so that the registration thread will not starve.

        Attachments

        1. HADOOP-15486.000.patch
          7 kB
          Nanda kumar
        2. HADOOP-15486.001.patch
          0.8 kB
          Nanda kumar

          Issue Links

            Activity

              People

              • Assignee:
                nanda Nanda kumar
                Reporter:
                nanda Nanda kumar
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: