HBase
  1. HBase
  2. HBASE-4240

Allow Loadbalancer to be pluggable.

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.94.0
    • Fix Version/s: 0.92.0
    • Component/s: master
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Everyone seems to want something different from a load balancer. People want low latency, simplicity, and total control. It seems like at some point the load balancer can't be all things to all people. Something akin to what hadoop JT's pluggable scheduler seems like it will enable all solutions without making the code much more complex.

      1. HBASE-4240-3.patch
        109 kB
        Elliott Clark
      2. HBASE-4240-2.patch
        110 kB
        Elliott Clark
      3. HBASE-4240-1.patch
        110 kB
        Elliott Clark
      4. HBASE-4240-0.patch
        84 kB
        Elliott Clark

        Issue Links

          Activity

          Harsh J made changes -
          Link This issue is blocked by HBASE-5143 [ HBASE-5143 ]
          Hide
          Harsh J added a comment -

          Hi,

          This introduced a badly named config. Please see HBASE-5143 for a fix.

          Show
          Harsh J added a comment - Hi, This introduced a badly named config. Please see HBASE-5143 for a fix.
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #2133 (See https://builds.apache.org/job/HBase-TRUNK/2133/)
          HBASE-4240 Allow Loadbalancer to be pluggable

          stack :
          Files :

          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/RegionPlan.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/LoadBalancerFactory.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/DefaultLoadBalancer.java
          • /hbase/trunk/CHANGES.txt
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ServerAndLoad.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestLoadBalancer.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestDefaultLoadBalancer.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK #2133 (See https://builds.apache.org/job/HBase-TRUNK/2133/ ) HBASE-4240 Allow Loadbalancer to be pluggable stack : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/RegionPlan.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/LoadBalancerFactory.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/DefaultLoadBalancer.java /hbase/trunk/CHANGES.txt /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ServerAndLoad.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestLoadBalancer.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestDefaultLoadBalancer.java
          stack made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Fix Version/s 0.92.0 [ 12314223 ]
          Fix Version/s 0.94.0 [ 12316419 ]
          Resolution Fixed [ 1 ]
          Hide
          stack added a comment -

          Nice patch Elliott. Nice review Ted. Yeah, that horror method needs refactoring but... another time. Committed to TRUNK.

          Show
          stack added a comment - Nice patch Elliott. Nice review Ted. Yeah, that horror method needs refactoring but... another time. Committed to TRUNK.
          Hide
          Ted Yu added a comment -

          I think balanceCluster() has got to a point where it is hard to maintain. Let alone add new features.
          Kudo to Elliot.

          Show
          Ted Yu added a comment - I think balanceCluster() has got to a point where it is hard to maintain. Let alone add new features. Kudo to Elliot.
          Hide
          stack added a comment -

          +1 Excellent. This helps a bunch.

          (Wah!!!! Elliott, you didn't have guts to refactor balanceCluster! – smile).

          Show
          stack added a comment - +1 Excellent. This helps a bunch. (Wah!!!! Elliott, you didn't have guts to refactor balanceCluster! – smile).
          Elliott Clark made changes -
          Attachment HBASE-4240-3.patch [ 12491305 ]
          Hide
          Elliott Clark added a comment -

          yeah no I idea what I thought that test method was testing.

          Show
          Elliott Clark added a comment - yeah no I idea what I thought that test method was testing.
          Hide
          Ted Yu added a comment -

          I am not sure how useful it is for testRandomizer() to test randomize() which resides only in the same test.
          I was saying that randomize() and testRandomizer() should both be removed.

          Show
          Ted Yu added a comment - I am not sure how useful it is for testRandomizer() to test randomize() which resides only in the same test. I was saying that randomize() and testRandomizer() should both be removed.
          Elliott Clark made changes -
          Attachment HBASE-4240-2.patch [ 12491304 ]
          Hide
          Elliott Clark added a comment -

          Added Javadoc to TestDefaultLoadBalancer.
          Moved Copyright on changed files to 2011
          Moved randomize to TestDefaultLoadBalancer.

          Show
          Elliott Clark added a comment - Added Javadoc to TestDefaultLoadBalancer. Moved Copyright on changed files to 2011 Moved randomize to TestDefaultLoadBalancer.
          Hide
          Ted Yu added a comment -
          + * Copyright 2010 The Apache Software Foundation
          

          The year should be 2011.

          +
          +public class TestDefaultLoadBalancer {
          

          Please add javadoc for the above class.

          Show
          Ted Yu added a comment - + * Copyright 2010 The Apache Software Foundation The year should be 2011. + + public class TestDefaultLoadBalancer { Please add javadoc for the above class.
          Hide
          Ted Yu added a comment -
          +  public List<HRegionInfo> randomize(final List<HRegionInfo> regions) {
          

          The only reference to the above method I can find is in TestLoadBalancer.

          I think it is safe to remove it from interface and implementation.

          Show
          Ted Yu added a comment - + public List<HRegionInfo> randomize( final List<HRegionInfo> regions) { The only reference to the above method I can find is in TestLoadBalancer. I think it is safe to remove it from interface and implementation.
          Elliott Clark made changes -
          Attachment HBASE-4240-1.patch [ 12491302 ]
          Hide
          Elliott Clark added a comment -

          Added more docs and changed the key.

          Show
          Elliott Clark added a comment - Added more docs and changed the key.
          Hide
          Ted Yu added a comment -

          TestLoadBalancer should be named TestDefaultLoadBalancer since factory isn't used to generate the balancer.

          A better name for "hbase.maser.loadBalancer" would be "hbase.maser.loadbalancer.class"

          Show
          Ted Yu added a comment - TestLoadBalancer should be named TestDefaultLoadBalancer since factory isn't used to generate the balancer. A better name for "hbase.maser.loadBalancer" would be "hbase.maser.loadbalancer.class"
          Hide
          Elliott Clark added a comment -

          Attached patch. It's not well polished right now. It needs some javadoc love and I need to add some other tests to show how to plug. Comments are welcomed.

          Show
          Elliott Clark added a comment - Attached patch. It's not well polished right now. It needs some javadoc love and I need to add some other tests to show how to plug. Comments are welcomed.
          Elliott Clark made changes -
          Field Original Value New Value
          Attachment HBASE-4240-0.patch [ 12491292 ]
          Elliott Clark created issue -

            People

            • Assignee:
              Elliott Clark
              Reporter:
              Elliott Clark
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development