Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15486

Make NetworkTopology#netLock fair

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      Whenever a datanode is restarted, the registration call after the restart received by NameNode lands in NetworkTopology#add via DatanodeManager#registerDatanode requires write lock on NetworkTopology#netLock.

      This registration thread is getting starved by flood of FSNamesystem.getAdditionalDatanode calls, which are triggered by clients those who were writing to the restarted datanode.

      The registration call which is waiting for write lock on NetworkTopology#netLock is holding write lock on FSNamesystem#fsLock, causing all the other RPC calls which require the lock on FSNamesystem#fsLock wait.
      We can make NetworkTopology#netLock lock fair so that the registration thread will not starve.

      Attachments

        1. HADOOP-15486.001.patch
          0.8 kB
          Nandakumar
        2. HADOOP-15486.000.patch
          7 kB
          Nandakumar

        Issue Links

          Activity

            People

              nanda Nandakumar
              nanda Nandakumar
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: