Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-23949

refactor loadBalancer implements for rsgroup balance by table to achieve overallbalanced

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 3.0.0-alpha-1, 2.3.0, 2.2.5
    • Component/s: rsgroup
    • Labels:
      None

      Description

       now can not achieve overallbalanced when use rsgroup balancer and by table is on,

      because balance every table actually use the clusterload only contain one table's load.

      we should use clusterload contain all this rsgroup table's load to balance overall

       

       hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java

        public boolean balance(boolean force) throws IOException {
          ......
          boolean isByTable = getConfiguration().getBoolean("hbase.master.loadbalance.bytable", false);
          Map<TableName, Map<ServerName, List<RegionInfo>>> assignments =
            this.assignmentManager.getRegionStates()
              .getAssignmentsForBalancer(tableStateManager, this.serverManager.getOnlineServersList(),
                isByTable);
          for (Map<ServerName, List<RegionInfo>> serverMap : assignments.values()) {
            serverMap.keySet().removeAll(this.serverManager.getDrainingServersList());
          }
      
      //Give the balancer the current cluster state.
          this.balancer.setClusterMetrics(getClusterMetricsWithoutCoprocessor());
          this.balancer.setClusterLoad(assignments);
      
          List<RegionPlan> plans = new ArrayList<>();
          for (Entry<TableName, Map<ServerName, List<RegionInfo>>> e : assignments.entrySet()) {
            List<RegionPlan> partialPlans = this.balancer.balanceCluster(e.getKey(), e.getValue());
            if (partialPlans != null) {
              plans.addAll(partialPlans);
            }
          }
      

      now do refactor:

      1. add method 'balanceTable' in interface LoadBalancer
      2. SimpleLoadBalancer and StochasticLoadBalancer do the real 'balanceTable' , and 'balanceTable' is not support in BaseLoadBalancer and RSGroupBasedLoadBalancer
      3. RSGroupBasedLoadBalancer invoke balanceCluster , and pass GroupClusterLoad to internal balacer by group
      4. internal balancer balance cluster invoke 'balanceTable' 

        Attachments

          Activity

            People

            • Assignee:
              niuyulin niuyulin
              Reporter:
              niuyulin niuyulin
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: