Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1312 Re-balance disks within a Datanode
  3. HDFS-9720

DiskBalancer : Add configuration parameters

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: HDFS-1312
    • Fix Version/s: HDFS-1312
    • Component/s: balancer & mover
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Add the following config params:

      1. Max Disk Throughput - This allows users to control how much disk I/O is generated while disk balancer is running
      2. Max Disk Errors - For each move operation, this allows user to specify how many I/O failures are tolerated before we declare the operation as failed.
        #Block Tolerance - Specifies how much movement of data is good enough. since the datanodes are active while the copy is going on, tolerance lets the user specify that if we reached something like 5% of projected Ideal storage on the destination disk, we can consider that move to be successful.

      Each of these parameter can be specified for each MoveStep. if not specified then the config parameters are used.

        Attachments

        1. HDFS-9720-HDFS-1312.001.patch
          37 kB
          Anu Engineer
        2. HDFS-9720-HDFS-1312.002.patch
          36 kB
          Anu Engineer
        3. HDFS-9720-HDFS-1312.003.patch
          36 kB
          Anu Engineer

          Activity

            People

            • Assignee:
              aengineer Anu Engineer
              Reporter:
              aengineer Anu Engineer
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: