HBase
  1. HBase
  2. HBASE-5970

Improve the AssignmentManager#updateTimer and speed up handling opened event

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.95.0
    • Component/s: master
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We found handing opened event very slow in the environment with lots of regions.

      The problem is the slow AssignmentManager#updateTimer.

      We do the test for bulk assigning 10w (i.e. 100k) regions, the whole process of bulk assigning took 1 hours.

      2012-05-06 20:31:49,201 INFO org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 100000 region(s) round-robin across 5 server(s)
      2012-05-06 21:26:32,103 INFO org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done

      I think we could do the improvement for the AssignmentManager#updateTimer: Make a thread do this work.
      After the improvement, it took only 4.5mins

      2012-05-07 11:03:36,581 INFO org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 100000 region(s) across 5 server(s), retainAssignment=true
      2012-05-07 11:07:57,073 INFO org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done

      1. HBASE-5970v4.patch
        4 kB
        chunhui shen
      2. HBASE-5970v4.patch
        4 kB
        chunhui shen
      3. 5970v3.patch
        4 kB
        Ted Yu
      4. HBASE-5970v3.patch
        4 kB
        chunhui shen
      5. HBASE-5970v2.patch
        3 kB
        chunhui shen
      6. HBASE-5970.patch
        4 kB
        chunhui shen

        Issue Links

          Activity

            People

            • Assignee:
              chunhui shen
              Reporter:
              chunhui shen
            • Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development