Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-5970

Improve the AssignmentManager#updateTimer and speed up handling opened event

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.95.0
    • Component/s: master
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We found handing opened event very slow in the environment with lots of regions.

      The problem is the slow AssignmentManager#updateTimer.

      We do the test for bulk assigning 10w (i.e. 100k) regions, the whole process of bulk assigning took 1 hours.

      2012-05-06 20:31:49,201 INFO org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 100000 region(s) round-robin across 5 server(s)
      2012-05-06 21:26:32,103 INFO org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done

      I think we could do the improvement for the AssignmentManager#updateTimer: Make a thread do this work.
      After the improvement, it took only 4.5mins

      2012-05-07 11:03:36,581 INFO org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 100000 region(s) across 5 server(s), retainAssignment=true
      2012-05-07 11:07:57,073 INFO org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done

        Attachments

        1. HBASE-5970.patch
          4 kB
          chunhui shen
        2. HBASE-5970v2.patch
          3 kB
          chunhui shen
        3. HBASE-5970v3.patch
          4 kB
          chunhui shen
        4. 5970v3.patch
          4 kB
          Ted Yu
        5. HBASE-5970v4.patch
          4 kB
          chunhui shen
        6. HBASE-5970v4.patch
          4 kB
          chunhui shen

          Issue Links

            Activity

              People

              • Assignee:
                zjushch chunhui shen
                Reporter:
                zjushch chunhui shen
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: