HBase
  1. HBase
  2. HBASE-1676

load balancing on a large cluster doesn't work very well

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.0
    • Fix Version/s: 0.90.0
    • Component/s: None
    • Labels:
      None

      Description

      I have 200 regions per server in my cluster. When I take one down, then bring it back up, I only end up with about 30 regions assigned to it. This is highly reproducible.

      Furthermore, the assignment algorithm will do silly things like assign all regions of a table to one regionserver - even if it's the best server. We need to be smarter about optimizing how regions are assigned if we want to achieve maximal performance on our clusters.

        Issue Links

          Activity

          Hide
          stack added a comment -

          Lets move the assignment code to its own class to facilitate testing alogrithm in isolation. Then write one million unit tests for it.

          Show
          stack added a comment - Lets move the assignment code to its own class to facilitate testing alogrithm in isolation. Then write one million unit tests for it.
          Hide
          Bryan Duxbury added a comment -

          +1 to that idea.

          Show
          Bryan Duxbury added a comment - +1 to that idea.
          Hide
          stack added a comment -

          Too big to fix in a point release. Moving to 0.21.

          Show
          stack added a comment - Too big to fix in a point release. Moving to 0.21.
          Hide
          stack added a comment -

          Master Rewrite should make it so assignment is testable outside of region context.

          Show
          stack added a comment - Master Rewrite should make it so assignment is testable outside of region context.
          Hide
          Jonathan Gray added a comment -

          There are standalone tests of the new load balancer! This is open against 0.90. Do we want to close it or what exactly is the scope of this jira? IMO this is done, other stuff can be handled in "smart" load balancing and other open jiras.

          Show
          Jonathan Gray added a comment - There are standalone tests of the new load balancer! This is open against 0.90. Do we want to close it or what exactly is the scope of this jira? IMO this is done, other stuff can be handled in "smart" load balancing and other open jiras.
          Hide
          stack added a comment -

          Resolving as done as part of master rewrite issue. Open new issue if the new load balancer committed there is unsatisfactory.

          Show
          stack added a comment - Resolving as done as part of master rewrite issue. Open new issue if the new load balancer committed there is unsatisfactory.

            People

            • Assignee:
              Unassigned
              Reporter:
              ryan rawson
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development