Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20907

Fix Intermittent failure on TestProcedurePriority

    XMLWordPrintableJSON

Details

    • Test
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0-alpha-1, 2.2.0, 2.1.1, 2.0.2
    • None
    • None
    • Reviewed

    Description

      From a local UT check against 2.1.0-RC1, HMaster failed to initialize before time out. Checking the test log we could see below message:

      2018-07-17 20:06:37,142 DEBUG [Thread-4003] client.RpcRetryingCallerImpl(131): Call exception, tries=6, retries=6, started=4173 ms ago, cancelled=false, msg=java.io.IOException: Inject error
              at org.apache.hadoop.hbase.master.procedure.TestProcedurePriority$MyCP.preGetOp(TestProcedurePriority.java:92)
              at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:841)
              at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:838)
              at org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:540)
              at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:614)
              at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:838)
              at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2520)
              at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2460)
              at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
              at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
              at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
              at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
              at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
      , details=row 'hbase:namespace' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=hdpdevm1.et2sqa.tbsite.net,59254,1531829189215, seqNum=-1, exception=java.io.IOException: java.io.IOException: Inject error
              at org.apache.hadoop.hbase.master.procedure.TestProcedurePriority$MyCP.preGetOp(TestProcedurePriority.java:92)
              at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:841)
              ...
              at org.apache.hadoop.hbase.client.HTable.get(HTable.java:386)
              at org.apache.hadoop.hbase.client.HTable.get(HTable.java:360)
              at org.apache.hadoop.hbase.MetaTableAccessor.getTableState(MetaTableAccessor.java:1078)
              at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:403)
              at org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:94)
      

      In current test code we will set FAIL to true w/o checking whether namespace manager is already up, and if not lucky we will run into the above case and get a timeout.

      The fix will be quite straight forward.

      Attachments

        1. HBASE-20907.patch
          1 kB
          Yu Li
        2. HBASE-20907.v2.patch
          1 kB
          Yu Li

        Activity

          People

            liyu Yu Li
            liyu Yu Li
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: