Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-21213

[hbck2] bypass leaves behind state in RegionStates when assign/unassign

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.1.1
    • Component/s: amv2, hbck2
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Adds override to assigns and unassigns. Changes bypass 'force' to align calling the param 'override' instead.

      The new override allows 'overriding' previous Procedure owner to override their ownership of the Procedured entity ("Region").

      Used by hbck2.
      Show
      Adds override to assigns and unassigns. Changes bypass 'force' to align calling the param 'override' instead. The new override allows 'overriding' previous Procedure owner to override their ownership of the Procedured entity ("Region"). Used by hbck2.

      Description

      This is a follow-on from HBASE-21083 which added the 'bypass' functionality. On bypass, there is more state to be cleared if we are allow new Procedures to be scheduled.

      For example, here is a bypass:

      2018-09-20 05:45:43,722 INFO org.apache.hadoop.hbase.procedure2.Procedure: pid=100449, state=RUNNABLE:REGION_TRANSITION_DISPATCH, locked=true, bypass=LOG-REDACTED UnassignProcedure table=hbase:namespace, region=37cc206fe9c4bc1c0a46a34c5f523d16, server=ve1233.halxg.cloudera.com,22101,1537397961664 bypassed, returning null to finish it
      2018-09-20 05:45:44,022 INFO org.apache.hadoop.hbase.procedure2.ProcedureExecutor: Finished pid=100449, state=SUCCESS, bypass=LOG-REDACTED UnassignProcedure table=hbase:namespace, region=37cc206fe9c4bc1c0a46a34c5f523d16, server=ve1233.halxg.cloudera.com,22101,1537397961664 in 2mins, 7.618sec
      

      ... but then when I try to assign the bypassed region later, I get this:

      2018-09-20 05:46:31,435 WARN org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure: There is already another procedure running on this region this=pid=100450, state=RUNNABLE:REGION_TRANSITION_QUEUE, locked=true; AssignProcedure table=hbase:namespace, region=37cc206fe9c4bc1c0a46a34c5f523d16 owner=pid=100449, state=SUCCESS, bypass=LOG-REDACTED UnassignProcedure table=hbase:namespace, region=37cc206fe9c4bc1c0a46a34c5f523d16, server=ve1233.halxg.cloudera.com,22101,1537397961664 pid=100450, state=RUNNABLE:REGION_TRANSITION_QUEUE, locked=true; AssignProcedure table=hbase:namespace, region=37cc206fe9c4bc1c0a46a34c5f523d16; rit=OPENING, location=ve1233.halxg.cloudera.com,22101,1537397961664
      2018-09-20 05:46:31,510 INFO org.apache.hadoop.hbase.procedure2.ProcedureExecutor: Rolled back pid=100450, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via AssignProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: There is already another procedure running on this region this=pid=100450, state=RUNNABLE:REGION_TRANSITION_QUEUE, locked=true; AssignProcedure table=hbase:namespace, region=37cc206fe9c4bc1c0a46a34c5f523d16 owner=pid=100449, state=SUCCESS, bypass=LOG-REDACTED UnassignProcedure table=hbase:namespace, region=37cc206fe9c4bc1c0a46a34c5f523d16, server=ve1233.halxg.cloudera.com,22101,1537397961664; AssignProcedure table=hbase:namespace, region=37cc206fe9c4bc1c0a46a34c5f523d16 exec-time=473msec
      

      ... which is a long-winded way of saying the Unassign Procedure still exists still in RegionStateNodes.

        Attachments

        1. HBASE-21213.branch-2.1.011.patch
          70 kB
          Michael Stack
        2. HBASE-21213.branch-2.1.010.patch
          70 kB
          Michael Stack
        3. HBASE-21213.branch-2.1.009.patch
          69 kB
          Michael Stack
        4. HBASE-21213.branch-2.1.008.patch
          62 kB
          Michael Stack
        5. HBASE-21213.branch-2.1.007.patch
          61 kB
          Michael Stack
        6. HBASE-21213.branch-2.1.007.patch
          61 kB
          Michael Stack
        7. HBASE-21213.branch-2.1.006.patch
          61 kB
          Michael Stack
        8. HBASE-21213.branch-2.1.005.patch
          59 kB
          Michael Stack
        9. HBASE-21213.branch-2.1.004.patch
          54 kB
          Michael Stack
        10. HBASE-21213.branch-2.1.003.patch
          54 kB
          Michael Stack
        11. HBASE-21213.branch-2.1.002.patch
          54 kB
          Michael Stack
        12. HBASE-21213.branch-2.1.001.patch
          23 kB
          Michael Stack

          Issue Links

            Activity

              People

              • Assignee:
                stack Michael Stack
                Reporter:
                stack Michael Stack
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: