[HBASE-20013] TestZKPermissionWatcher is flakey

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0-beta-2, 2.0.0
• Component/s: test
• Assignee: Michael Stack
• Reporter: Michael Stack
• Created: 17/Feb/18 04:03
• Updated: 07/May/19 16:08
• Resolved: 20/Feb/18 14:30

Description

The last two nightlies failed on this test in here on shutdown:

2018-02-16 20:49:35,132 DEBUG [M:0;881c50037eea:35808] master.MasterRpcServices(1153): Checking to see if procedure is done pid=7
2018-02-16 20:49:35,133 DEBUG [M:0;881c50037eea:35808] client.RpcRetryingCallerImpl(132): Call exception, tries=7, retries=7, started=8122 ms ago, cancelled=false, msg=null, details=, exception=org.apache.hadoop.hbase.MasterNotRunningException
  at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2736)
  at org.apache.hadoop.hbase.master.MasterRpcServices.getProcedureResult(MasterRpcServices.java:1155)
  at org.apache.hadoop.hbase.client.ShortCircuitMasterConnection.getProcedureResult(ShortCircuitMasterConnection.java:423)
  at org.apache.hadoop.hbase.client.HBaseAdmin$ProcedureFuture$2.rpcCall(HBaseAdmin.java:3490)
  at org.apache.hadoop.hbase.client.HBaseAdmin$ProcedureFuture$2.rpcCall(HBaseAdmin.java:3487)
  at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:100)
  at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:107)
  at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3055)
  at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3047)
  at org.apache.hadoop.hbase.client.HBaseAdmin.access$700(HBaseAdmin.java:224)
  at org.apache.hadoop.hbase.client.HBaseAdmin$ProcedureFuture.getProcedureResult(HBaseAdmin.java:3486)
  at org.apache.hadoop.hbase.client.HBaseAdmin$ProcedureFuture.waitProcedureResult(HBaseAdmin.java:3438)
  at org.apache.hadoop.hbase.client.HBaseAdmin$ProcedureFuture.get(HBaseAdmin.java:3394)
  at org.apache.hadoop.hbase.client.HBaseAdmin.get(HBaseAdmin.java:2123)
  at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:612)
  at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:586)
  at org.apache.hadoop.hbase.security.access.AccessController.createACLTable(AccessController.java:1130)
  at org.apache.hadoop.hbase.security.access.AccessController.postStartMaster(AccessController.java:1107)
  at org.apache.hadoop.hbase.master.MasterCoprocessorHost$71.call(MasterCoprocessorHost.java:994)
  at org.apache.hadoop.hbase.master.MasterCoprocessorHost$71.call(MasterCoprocessorHost.java:991)
  at org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:540)
  at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:614)
  at org.apache.hadoop.hbase.master.MasterCoprocessorHost.postStartMaster(MasterCoprocessorHost.java:991)
  at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:969)
  at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2026)
  at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:555)
  at java.lang.Thread.run(Thread.java:748)

We get stuck retrying.

Issue Links

• links to: Review Board (branch-2)