Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20186

Improve RSGroupBasedLoadBalancer#balanceCluster() to be more efficient when calculating cluster state for each rsgroup

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0, 2.1.0, 1.4.3
    • Component/s: rsgroup
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In RSGroupBasedLoadBalancer

      public List<RegionPlan> balanceCluster(Map<ServerName, List<RegionInfo>> clusterState)
      

      The second half of the function is to calculate region move plan for regions which have been already placed according to the rsgroup assignment, and it is calculated one rsgroup after another.
      The following logic to check if a server belongs to the rsgroup is not quite efficient, as it does not make good use of the fact that servers in RSGroupInfo is a TreeSet.

      for (Address sName : info.getServers()) {
        for(ServerName curr: clusterState.keySet()) {
          if(curr.getAddress().equals(sName)) {
            groupClusterState.put(curr, correctedState.get(curr));
          }
        }
      }
      

      Given there are m region servers in the cluster and n region servers for each rsgroup in average, the code above has time complexity as O(m * n), while using TreeSet's contains(), the time complexity could be reduced to O (m * logn).

      Another improvement is we do not need to scan every server for each rsgroup. If the processed server could be recorded, we could skip those.

        Attachments

        1. HBASE-20186.master.001.patch
          3 kB
          Xiang Li
        2. HBASE-20186.master.000.patch
          2 kB
          Xiang Li

          Activity

            People

            • Assignee:
              xiangli Xiang Li
              Reporter:
              xiangli Xiang Li
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: