Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20907

Fix Intermittent failure on TestProcedurePriority

    XMLWordPrintableJSON

    Details

    • Type: Test
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha-1, 2.2.0, 2.1.1, 2.0.2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      From a local UT check against 2.1.0-RC1, HMaster failed to initialize before time out. Checking the test log we could see below message:

      2018-07-17 20:06:37,142 DEBUG [Thread-4003] client.RpcRetryingCallerImpl(131): Call exception, tries=6, retries=6, started=4173 ms ago, cancelled=false, msg=java.io.IOException: Inject error
              at org.apache.hadoop.hbase.master.procedure.TestProcedurePriority$MyCP.preGetOp(TestProcedurePriority.java:92)
              at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:841)
              at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:838)
              at org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:540)
              at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:614)
              at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:838)
              at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2520)
              at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2460)
              at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
              at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
              at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
              at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
              at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
      , details=row 'hbase:namespace' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=hdpdevm1.et2sqa.tbsite.net,59254,1531829189215, seqNum=-1, exception=java.io.IOException: java.io.IOException: Inject error
              at org.apache.hadoop.hbase.master.procedure.TestProcedurePriority$MyCP.preGetOp(TestProcedurePriority.java:92)
              at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:841)
              ...
              at org.apache.hadoop.hbase.client.HTable.get(HTable.java:386)
              at org.apache.hadoop.hbase.client.HTable.get(HTable.java:360)
              at org.apache.hadoop.hbase.MetaTableAccessor.getTableState(MetaTableAccessor.java:1078)
              at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:403)
              at org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:94)
      

      In current test code we will set FAIL to true w/o checking whether namespace manager is already up, and if not lucky we will run into the above case and get a timeout.

      The fix will be quite straight forward.

        Attachments

        1. HBASE-20907.patch
          1 kB
          Yu Li
        2. HBASE-20907.v2.patch
          1 kB
          Yu Li

          Activity

            People

            • Assignee:
              liyu Yu Li
              Reporter:
              liyu Yu Li
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: