Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-6133

Make Balancer support exclude specified path

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.7.0
    • Component/s: balancer & mover, datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Add a feature for replica pinning so that when a replica is pinned in a datanode, it will not be moved by Balancer/Mover. The replica pinning feature can be enabled/disabled by "dfs.datanode.block-pinning.enabled", where the default is false.

      Description

      Currently, run Balancer will destroying Regionserver's data locality.
      If getBlocks could exclude blocks belongs to files which have specific path prefix, like "/hbase", then we can run Balancer without destroying Regionserver's data locality.

      1. HDFS-6133-9.patch
        37 kB
        zhaoyunjiong
      2. HDFS-6133-8.patch
        30 kB
        zhaoyunjiong
      3. HDFS-6133-7.patch
        30 kB
        zhaoyunjiong
      4. HDFS-6133-6.patch
        24 kB
        zhaoyunjiong
      5. HDFS-6133-5.patch
        23 kB
        zhaoyunjiong
      6. HDFS-6133-4.patch
        23 kB
        zhaoyunjiong
      7. HDFS-6133-3.patch
        22 kB
        zhaoyunjiong
      8. HDFS-6133-2.patch
        29 kB
        zhaoyunjiong
      9. HDFS-6133-11.patch
        33 kB
        zhaoyunjiong
      10. HDFS-6133-10.patch
        33 kB
        zhaoyunjiong
      11. HDFS-6133-1.patch
        29 kB
        zhaoyunjiong
      12. HDFS-6133.patch
        24 kB
        zhaoyunjiong

        Issue Links

          Activity

          zhaoyunjiong created issue -
          zhaoyunjiong made changes -
          Field Original Value New Value
          Attachment HDFS-6133.patch [ 12635897 ]
          zhaoyunjiong made changes -
          Summary Make Balancer support don't move blocks belongs to Hbase Make Balancer support exclude specified path
          zhaoyunjiong made changes -
          Attachment HDFS-6133.patch [ 12635897 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133.patch [ 12636315 ]
          zhaoyunjiong made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133.patch [ 12641679 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133.patch [ 12636315 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133.patch [ 12641891 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133.patch [ 12641679 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133.patch.1 [ 12643533 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133.patch.1 [ 12643533 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133-1.patch [ 12643535 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133-2.patch [ 12653743 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133-3.patch [ 12673853 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133-3.patch [ 12673853 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133-3.patch [ 12674125 ]
          Chris Trezzo made changes -
          Link This issue is related to HDFS-4420 [ HDFS-4420 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133-4.patch [ 12682349 ]
          Devaraj Das made changes -
          Link This issue relates to HBASE-4755 [ HBASE-4755 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133-5.patch [ 12683784 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133-6.patch [ 12688622 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133-7.patch [ 12696748 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133-7.patch [ 12696748 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133-7.patch [ 12696984 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133-8.patch [ 12697036 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133-9.patch [ 12697420 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133-10.patch [ 12697628 ]
          Tsz Wo Nicholas Sze made changes -
          Hadoop Flags Reviewed [ 10343 ]
          zhaoyunjiong made changes -
          Link This issue relates to HDFS-7759 [ HDFS-7759 ]
          zhaoyunjiong made changes -
          Attachment HDFS-6133-11.patch [ 12697963 ]
          Tsz Wo Nicholas Sze made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 2.7.0 [ 12327584 ]
          Resolution Fixed [ 1 ]
          Tsz Wo Nicholas Sze made changes -
          Component/s datanode [ 12312927 ]
          Component/s namenode [ 12312926 ]
          Tsz Wo Nicholas Sze made changes -
          Release Note Add a feature for replica pinning so that a pinned replica will not be moved by Balancer/Mover. The replica pinning feature can be enabled/disabled by dfs.datanode.block-pinning.enabled, where the default is false.
          Tsz Wo Nicholas Sze made changes -
          Release Note Add a feature for replica pinning so that a pinned replica will not be moved by Balancer/Mover. The replica pinning feature can be enabled/disabled by dfs.datanode.block-pinning.enabled, where the default is false. Add a feature for replica pinning so that when a replica is pinned in a datanode, it will not be moved by Balancer/Mover. The replica pinning feature can be enabled/disabled by "dfs.datanode.block-pinning.enabled", where the default is false.
          Tsz Wo Nicholas Sze made changes -
          Link This issue relates to HDFS-7849 [ HDFS-7849 ]
          Tsz Wo Nicholas Sze made changes -
          Link This issue breaks HDFS-7954 [ HDFS-7954 ]
          Vinod Kumar Vavilapalli made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              zhaoyunjiong
              Reporter:
              zhaoyunjiong
            • Votes:
              1 Vote for this issue
              Watchers:
              32 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development