Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-19828

Flakey TestRegionsOnMasterOptions.testRegionsOnAllServers

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Disables TestRegionsOnMasterOptions because Regions on Master does not work reliably; see HBASE-19831.

      Description

      This test is failing 50% of the time now. We seem to have made it fail more w/ our recent changes. The failure here is a good one, again, a real issue. We can get stuck trying to update meta with a Region state if the cluster is going down and hbase:meta has gone down before our client; we get locked-up retrying the put to hbase:meta for so long, the test times out:

       

      Thread 2080 (ProcExecWrkr-7):
      State: TIMED_WAITING
      Blocked count: 0
      Waited count: 5960
      Stack:
      java.lang.Object.wait(Native Method)
      org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.waitUntilDone(AsyncRequestFutureImpl.java:1228)
      org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.waitUntilDone(AsyncRequestFutureImpl.java:1197)
      org.apache.hadoop.hbase.client.HTable.doBatchWithCallback(HTable.java:485)
      org.apache.hadoop.hbase.util.MultiHConnection.processBatchCallback(MultiHConnection.java:122)
      org.apache.hadoop.hbase.master.assignment.RegionStateStore.updateRegionLocation(RegionStateStore.java:222)
      org.apache.hadoop.hbase.master.assignment.RegionStateStore.updateUserRegionLocation(RegionStateStore.java:209)
      org.apache.hadoop.hbase.master.assignment.RegionStateStore.updateRegionLocation(RegionStateStore.java:149)
      org.apache.hadoop.hbase.master.assignment.AssignmentManager.markRegionAsClosing(AssignmentManager.java:1536)
      org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:179)
      org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:309)
      org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:85)
      org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
      org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1456)
      org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1225)
      org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
      org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1736)

        Attachments

        1. HBASE-19828.master.001.patch
          6 kB
          Michael Stack

          Issue Links

            Activity

              People

              • Assignee:
                stack Michael Stack
                Reporter:
                stack Michael Stack
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: