------------------------------------------------------------------------------- Test set: org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClientAfterSplittingRegions ------------------------------------------------------------------------------- Tests run: 6, Failures: 0, Errors: 6, Skipped: 0, Time elapsed: 780.997 s <<< FAILURE! - in org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClientAfterSplittingRegions testRestoreSnapshotAfterSplittingRegions[0: regionReplication=1](org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClientAfterSplittingRegions) Time elapsed: 463.104 s <<< ERROR! org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=7, exceptions: Tue Dec 04 20:50:33 UTC 2018, RpcRetryingCaller{globalStartTime=1543956573840, pause=250, maxAttempts=7}, java.io.IOException: Call to asf910.gq1.ygridcore.net/67.195.81.154:53736 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=141, waitTime=60006, rpcTimeout=60000 Tue Dec 04 20:51:34 UTC 2018, RpcRetryingCaller{globalStartTime=1543956573840, pause=250, maxAttempts=7}, java.io.IOException: Call to asf910.gq1.ygridcore.net/67.195.81.154:53736 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=143, waitTime=60004, rpcTimeout=60000 Tue Dec 04 20:52:34 UTC 2018, RpcRetryingCaller{globalStartTime=1543956573840, pause=250, maxAttempts=7}, java.io.IOException: Call to asf910.gq1.ygridcore.net/67.195.81.154:53736 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=145, waitTime=60005, rpcTimeout=60000 Tue Dec 04 20:53:35 UTC 2018, RpcRetryingCaller{globalStartTime=1543956573840, pause=250, maxAttempts=7}, java.io.IOException: Call to asf910.gq1.ygridcore.net/67.195.81.154:53736 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=147, waitTime=60010, rpcTimeout=60000 Tue Dec 04 20:54:36 UTC 2018, RpcRetryingCaller{globalStartTime=1543956573840, pause=250, maxAttempts=7}, java.io.IOException: Call to asf910.gq1.ygridcore.net/67.195.81.154:53736 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=149, waitTime=60009, rpcTimeout=60000 Tue Dec 04 20:56:39 UTC 2018, RpcRetryingCaller{globalStartTime=1543956573840, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: java.io.IOException: Call to asf910.gq1.ygridcore.net/67.195.81.154:53736 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=151, waitTime=60008, rpcTimeout=60000 Tue Dec 04 20:56:44 UTC 2018, RpcRetryingCaller{globalStartTime=1543956573840, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Caused by: org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Caused by: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? testRestoreSnapshotAfterSplittingRegions[0: regionReplication=1](org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClientAfterSplittingRegions) Time elapsed: 463.105 s <<< ERROR! org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=7, exceptions: Tue Dec 04 20:56:44 UTC 2018, RpcRetryingCaller{globalStartTime=1543957004272, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Tue Dec 04 20:56:44 UTC 2018, RpcRetryingCaller{globalStartTime=1543957004272, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Tue Dec 04 20:56:45 UTC 2018, RpcRetryingCaller{globalStartTime=1543957004272, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Tue Dec 04 20:56:45 UTC 2018, RpcRetryingCaller{globalStartTime=1543957004272, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Tue Dec 04 20:56:47 UTC 2018, RpcRetryingCaller{globalStartTime=1543957004272, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Tue Dec 04 20:56:49 UTC 2018, RpcRetryingCaller{globalStartTime=1543957004272, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Tue Dec 04 20:56:54 UTC 2018, RpcRetryingCaller{globalStartTime=1543957004272, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Caused by: org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Caused by: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? testRestoreSnapshotAfterSplittingRegions[1: regionReplication=3](org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClientAfterSplittingRegions) Time elapsed: 20.759 s <<< ERROR! org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=7, exceptions: Tue Dec 04 20:56:54 UTC 2018, RpcRetryingCaller{globalStartTime=1543957014749, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Tue Dec 04 20:56:55 UTC 2018, RpcRetryingCaller{globalStartTime=1543957014749, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Tue Dec 04 20:56:55 UTC 2018, RpcRetryingCaller{globalStartTime=1543957014749, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Tue Dec 04 20:56:56 UTC 2018, RpcRetryingCaller{globalStartTime=1543957014749, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Tue Dec 04 20:56:57 UTC 2018, RpcRetryingCaller{globalStartTime=1543957014749, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Tue Dec 04 20:57:00 UTC 2018, RpcRetryingCaller{globalStartTime=1543957014749, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Tue Dec 04 20:57:05 UTC 2018, RpcRetryingCaller{globalStartTime=1543957014749, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Caused by: org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Caused by: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? testRestoreSnapshotAfterSplittingRegions[1: regionReplication=3](org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClientAfterSplittingRegions) Time elapsed: 20.76 s <<< ERROR! org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=7, exceptions: Tue Dec 04 20:57:05 UTC 2018, RpcRetryingCaller{globalStartTime=1543957025108, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Tue Dec 04 20:57:05 UTC 2018, RpcRetryingCaller{globalStartTime=1543957025108, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Tue Dec 04 20:57:05 UTC 2018, RpcRetryingCaller{globalStartTime=1543957025108, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Tue Dec 04 20:57:06 UTC 2018, RpcRetryingCaller{globalStartTime=1543957025108, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Tue Dec 04 20:57:07 UTC 2018, RpcRetryingCaller{globalStartTime=1543957025108, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Tue Dec 04 20:57:10 UTC 2018, RpcRetryingCaller{globalStartTime=1543957025108, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Tue Dec 04 20:57:15 UTC 2018, RpcRetryingCaller{globalStartTime=1543957025108, pause=250, maxAttempts=7}, org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Caused by: org.apache.hadoop.hbase.MasterNotRunningException: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Caused by: org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException: Call queue is full on asf910.gq1.ygridcore.net,53736,1543956537196, too many items queued ? org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClientAfterSplittingRegions Time elapsed: 293.628 s <<< ERROR! org.junit.runners.model.TestTimedOutException: test timed out after 780 seconds org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClientAfterSplittingRegions Time elapsed: 293.651 s <<< ERROR! java.lang.Exception: Appears to be stuck in thread RS-EventLoopGroup-5-3