Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14859

Prevent unnecessary evaluation of costly operation getNumLiveDataNodes when dfs.namenode.safemode.min.datanodes is not zero

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.1.0, 3.3.0, 3.1.4
    • Fix Version/s: 3.3.0, 3.1.4, 3.2.2
    • Component/s: hdfs
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      There have been improvements like HDFS-14171 and HDFS-14632 to the performance issue caused from getNumLiveDataNodes calls per block. The improvement has been only done w.r.t dfs.namenode.safemode.min.datanodes paramter being set to 0 or not.

         private boolean areThresholdsMet() {
           assert namesystem.hasWriteLock();
      -    int datanodeNum = blockManager.getDatanodeManager().getNumLiveDataNodes();
      +    // Calculating the number of live datanodes is time-consuming
      +    // in large clusters. Skip it when datanodeThreshold is zero.
      +    int datanodeNum = 0;
      +    if (datanodeThreshold > 0) {
      +      datanodeNum = blockManager.getDatanodeManager().getNumLiveDataNodes();
      +    }
           synchronized (this) {
             return blockSafe >= blockThreshold && datanodeNum >= datanodeThreshold;
           }
      

      I feel above logic would create similar situation of un-necessary evaluations of getNumLiveDataNodes when dfs.namenode.safemode.min.datanodes paramter is set > 0 even though "blockSafe >= blockThreshold" is false for most of the time in NN startup safe mode. We could do something like below to avoid this

      private boolean areThresholdsMet() {
          assert namesystem.hasWriteLock();
          synchronized (this) {
            return blockSafe >= blockThreshold && (datanodeThreshold > 0)?
                    blockManager.getDatanodeManager().getNumLiveDataNodes() >= datanodeThreshold : true;
          }
        } 
      

        Attachments

        1. HDFS-14859.001.patch
          1 kB
          Srinivasu Majeti
        2. HDFS-14859.002.patch
          5 kB
          Srinivasu Majeti
        3. HDFS-14859.003.patch
          5 kB
          Srinivasu Majeti
        4. HDFS-14859.004.patch
          5 kB
          Srinivasu Majeti
        5. HDFS-14859.005.patch
          5 kB
          Srinivasu Majeti
        6. HDFS-14859.006.patch
          6 kB
          Srinivasu Majeti
        7. HDFS-14859.007.patch
          6 kB
          Srinivasu Majeti

          Issue Links

            Activity

              People

              • Assignee:
                smajeti Srinivasu Majeti
                Reporter:
                smajeti Srinivasu Majeti
              • Votes:
                1 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Due:
                  Created:
                  Updated:
                  Resolved: