Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-24925

SCP reduce unnecessary get requests

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.0.0-alpha-1, 2.2.3
    • Fix Version/s: None
    • Component/s: MTTR
    • Labels:
      None

      Description

      SCP should reduce unnecessary Get request

      https://github.com/apache/hbase/blob/65d28da7c22382e040363c607840d5ab6e6b45da/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java#L520

      during startup, the tableNam2State of tableStateManager is not loading tableState data form metaTable yet.  if procThread num is 50 and hbase has 10K tables, in the worst case, the master needs to query meta table 500K times(50*10K. and the regions that all SCPs simultaneously check tableState belong to the same table )

       

      i think master can reduce Get request, and AM#loadMeta can load regions and all tables through asynchronous threads.

      https://github.com/apache/hbase/blob/65d28da7c22382e040363c607840d5ab6e6b45da/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java#L1532

       

       

        Attachments

          Activity

            People

            • Assignee:
              Bo Cui Bo Cui
              Reporter:
              Bo Cui Bo Cui
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated: