Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-23949

refactor loadBalancer implements for rsgroup balance by table to achieve overallbalanced

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.2.0
    • 3.0.0-alpha-1, 2.3.0, 2.2.5
    • rsgroup
    • None

    Description

       now can not achieve overallbalanced when use rsgroup balancer and by table is on,

      because balance every table actually use the clusterload only contain one table's load.

      we should use clusterload contain all this rsgroup table's load to balance overall

       

       hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java

        public boolean balance(boolean force) throws IOException {
          ......
          boolean isByTable = getConfiguration().getBoolean("hbase.master.loadbalance.bytable", false);
          Map<TableName, Map<ServerName, List<RegionInfo>>> assignments =
            this.assignmentManager.getRegionStates()
              .getAssignmentsForBalancer(tableStateManager, this.serverManager.getOnlineServersList(),
                isByTable);
          for (Map<ServerName, List<RegionInfo>> serverMap : assignments.values()) {
            serverMap.keySet().removeAll(this.serverManager.getDrainingServersList());
          }
      
      //Give the balancer the current cluster state.
          this.balancer.setClusterMetrics(getClusterMetricsWithoutCoprocessor());
          this.balancer.setClusterLoad(assignments);
      
          List<RegionPlan> plans = new ArrayList<>();
          for (Entry<TableName, Map<ServerName, List<RegionInfo>>> e : assignments.entrySet()) {
            List<RegionPlan> partialPlans = this.balancer.balanceCluster(e.getKey(), e.getValue());
            if (partialPlans != null) {
              plans.addAll(partialPlans);
            }
          }
      

      now do refactor:

      1. add method 'balanceTable' in interface LoadBalancer
      2. SimpleLoadBalancer and StochasticLoadBalancer do the real 'balanceTable' , and 'balanceTable' is not support in BaseLoadBalancer and RSGroupBasedLoadBalancer
      3. RSGroupBasedLoadBalancer invoke balanceCluster , and pass GroupClusterLoad to internal balacer by group
      4. internal balancer balance cluster invoke 'balanceTable' 

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            niuyulin Yulin Niu
            niuyulin Yulin Niu
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment