Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14560

Allow block replication parameters to be refreshable

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.3.0
    • Fix Version/s: 3.3.0, 3.2.1, 3.1.3
    • Component/s: namenode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      There are 3 key parameters that control the speed of block replication across the cluster:

      dfs.namenode.replication.max-streams
      dfs.namenode.replication.max-streams-hard-limit
      dfs.namenode.replication.work.multiplier.per.iteration
      

      These are used when decomissioning nodes and when under replicated blocks are being recovered across the cluster. There are times when it may be desirable to increase the speed of replication and then reduce it again (eg during off peak hours) without restarting the namenode.

      Therefore it would be good to be able to reconfigure / refresh these parameters dynamically without a namenode restart.

      This Jira is to allow these parameters to be refreshed at runtime without a restart.

        Attachments

        1. HDFS-14560.branch-3.1.patch
          16 kB
          Stephen O'Donnell
        2. HDFS-14560.005.patch
          16 kB
          Stephen O'Donnell
        3. HDFS-14560.004.patch
          16 kB
          Stephen O'Donnell
        4. HDFS-14560.003.patch
          15 kB
          Stephen O'Donnell
        5. HDFS-14560.002.patch
          15 kB
          Stephen O'Donnell
        6. HDFS-14560.001.patch
          14 kB
          Stephen O'Donnell

          Activity

            People

            • Assignee:
              sodonnell Stephen O'Donnell
              Reporter:
              sodonnell Stephen O'Donnell
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: