[HBASE-13405] TestHBaseFsck is flaky

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Cannot Reproduce
• Affects Version/s: 2.0.0
• Component/s: test
• Assignee: Unassigned
• Reporter: Mikhail Antonov
• Created: 03/Apr/15 22:20
• Updated: 24/Jun/22 18:09
• Resolved: 24/May/18 16:43

Description

Once in a while I'm seeing the following, running #testContainedRegionOverlap test in IDE after clean install (mac osx, hbase master):

regionserver.HRegionServer(1863): Post open deploy tasks for tableContainedRegionOverlap,A,1428099123733.03a139b02119e99ef08149addd9a7996.
2015-04-03 15:12:11,695 INFO  [PostOpenDeployTasks:03a139b02119e99ef08149addd9a7996] regionserver.HRegionServer(1956): Failed to report region transition, will retry
java.io.InterruptedIOException: Origin: InterruptedException
	at org.apache.hadoop.hbase.util.ExceptionUtil.asInterrupt(ExceptionUtil.java:65)
	at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:313)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.reportRegionStateTransition(HRegionServer.java:1955)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1882)
	at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler$PostOpenDeployTasksThread.run(OpenRegionHandler.java:241)
Caused by: java.lang.InterruptedException: callId: 158 methodName: ReportRegionStateTransition param {TODO: class org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$ReportRegionStateTransitionRequest}
	at io.netty.util.concurrent.DefaultPromise.await0(DefaultPromise.java:333)
	at io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:266)
	at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:42)
	at org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:226)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
	at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.reportRegionStateTransition(RegionServerStatusProtos.java:9030)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.reportRegionStateTransition(HRegionServer.java:1946)
	... 2 more
2015-04-03 15:12:11,696 INFO  [B.defaultRpcServer.handler=1,queue=0,port=51217] master.MasterRpcServices(237): Client=mantonov//10.1.4.219 set balanceSwitch=false
2015-04-03 15:12:11,696 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(388): maste

and then:

015-04-03 15:12:11,796 INFO  [Thread-3019] client.HBaseAdmin$10(981): Started disable of tableContainedRegionOverlap
2015-04-03 15:12:21,641 INFO  [B.defaultRpcServer.handler=1,queue=0,port=51217] master.HMaster(1645): Client=mantonov//10.1.4.219 disable tableContainedRegionOverlap

java.lang.AssertionError: 
Expected :[]
Actual   :[NOT_DEPLOYED, HOLE_IN_REGION_CHAIN]
 <Click to see difference>
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotEquals(Assert.java:743)
	at org.junit.Assert.assertEquals(Assert.java:118)
	at org.junit.Assert.assertEquals(Assert.java:144)
	at org.apache.hadoop.hbase.util.hbck.HbckTestingUtil.assertNoErrors(HbckTestingUtil.java:92)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testContainedRegionOverlap(TestHBaseFsck.java:941)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)