Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-13103

[ergonomics] add region size balancing as a feature of master

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2.0, 1.3.0, 2.0.0
    • Component/s: Balancer, Usability
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      This patch adds optional ability for HMaster to normalize regions in size (disabled by default, change hbase.normalizer.enabled property to true to turn it on). If enabled, HMaster periodically (every 30 minutes by default) monitors tables for which normalization is enabled in table configuration and performs splits/merges as seems appropriate. Users may implement their own normalization strategies by implementing RegionNormalizer interface and configuring it in hbase-site.xml.
      Show
      This patch adds optional ability for HMaster to normalize regions in size (disabled by default, change hbase.normalizer.enabled property to true to turn it on). If enabled, HMaster periodically (every 30 minutes by default) monitors tables for which normalization is enabled in table configuration and performs splits/merges as seems appropriate. Users may implement their own normalization strategies by implementing RegionNormalizer interface and configuring it in hbase-site.xml.

      Description

      Often enough, folks miss-judge split points or otherwise end up with a suboptimal number of regions. We should have an automated, reliable way to "reshape" or "balance" a table's region boundaries. This would be for tables that contain existing data. This might look like:

      Admin#reshapeTable(TableName, int numSplits);
      

      or from the shell:

      > reshape TABLE, numSplits
      

      Better still would be to have a maintenance process, similar to the existing Balancer that runs AssignmentManager on an interval, to run the above "reshape" operation on an interval. That way, the cluster will automatically self-correct toward a desirable state.

        Attachments

        1. HBASE-13103-branch-1.v3.patch
          57 kB
          Mikhail Antonov
        2. HBASE-13103-v0.patch
          49 kB
          Mikhail Antonov
        3. HBASE-13103-v1.patch
          53 kB
          Mikhail Antonov
        4. HBASE-13103-v2.patch
          56 kB
          Mikhail Antonov
        5. HBASE-13103-v3.patch
          56 kB
          Mikhail Antonov

          Issue Links

            Activity

              People

              • Assignee:
                mantonov Mikhail Antonov
                Reporter:
                ndimiduk Nick Dimiduk
              • Votes:
                0 Vote for this issue
                Watchers:
                32 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: