Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7411

Refactor and improve decommissioning logic into DecommissionManager

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.5.1
    • Fix Version/s: 2.7.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      This change introduces a new configuration key used to throttle decommissioning work, "dfs.namenode.decommission.blocks.per.interval". This new key overrides and deprecates the previous related configuration key "dfs.namenode.decommission.nodes.per.interval". The new key is intended to result in more predictable pause times while scanning decommissioning nodes.

      Show
      This change introduces a new configuration key used to throttle decommissioning work, "dfs.namenode.decommission.blocks.per.interval". This new key overrides and deprecates the previous related configuration key "dfs.namenode.decommission.nodes.per.interval". The new key is intended to result in more predictable pause times while scanning decommissioning nodes.

      Description

      Would be nice to split out decommission logic from DatanodeManager to DecommissionManager.

        Attachments

        1. hdfs-7411.011.patch
          82 kB
          Andrew Wang
        2. hdfs-7411.010.patch
          81 kB
          Andrew Wang
        3. hdfs-7411.009.patch
          95 kB
          Andrew Wang
        4. hdfs-7411.008.patch
          94 kB
          Andrew Wang
        5. hdfs-7411.007.patch
          94 kB
          Andrew Wang
        6. hdfs-7411.006.patch
          89 kB
          Andrew Wang
        7. hdfs-7411.005.patch
          79 kB
          Andrew Wang
        8. hdfs-7411.004.patch
          78 kB
          Andrew Wang
        9. hdfs-7411.003.patch
          69 kB
          Andrew Wang
        10. hdfs-7411.002.patch
          24 kB
          Andrew Wang
        11. hdfs-7411.001.patch
          24 kB
          Andrew Wang

          Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. HDFS-12703 Exceptions are fatal to decommissioning monitor

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-5757 refreshNodes with many nodes at the same time could slow down NN

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-7442 Optimization for decommission-in-progress check

          • Major - Major loss of function.
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-8591 Remove support for deprecated configuration key dfs.namenode.decommission.nodes.per.interval

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

            Activity

              People

              • Assignee:
                andrew.wang Andrew Wang
                Reporter:
                andrew.wang Andrew Wang
              • Votes:
                0 Vote for this issue
                Watchers:
                21 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: