[HBASE-24480] Deflake TestRSGroupsBasics#testClearDeadServers

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.7.0
• Fix Version/s: 1.7.0
• Component/s: rsgroup
• Assignee: Bharath Vissapragada
• Reporter: Bharath Vissapragada
• Created: 01/Jun/20 01:48
• Updated: 02/Jun/20 10:41
• Resolved: 01/Jun/20 17:01

Description

Ran into this on our internal forks based on branch-1. It also applies to branch-2 but not master because the code has been re-implemented without co-proc due to HBASE-22514

Running into this exception in the test run..

org.apache.hadoop.hbase.constraint.ConstraintException: The set of servers to remove cannot be null or empty.
	at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.removeServers(RSGroupAdminServer.java:391)
	at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.postClearDeadServers(RSGroupAdminEndpoint.java:1175)
	at org.apache.hadoop.hbase.master.MasterCoprocessorHost$104.call(MasterCoprocessorHost.java:1251)
	at org.apache.hadoop.hbase.master.MasterCoprocessorHost.execOperation(MasterCoprocessorHost.java:1507)
	at org.apache.hadoop.hbase.master.MasterCoprocessorHost.postClearDeadServers(MasterCoprocessorHost.java:1247)
	at org.apache.hadoop.hbase.master.MasterRpcServices.clearDeadServers(MasterRpcServices.java:1167)
	at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2421)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:311)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:291)
" type="org.apache.hadoop.hbase.constraint.ConstraintException">org.apache.hadoop.hbase.constraint.ConstraintException: 
org.apache.hadoop.hbase.constraint.ConstraintException: The set of servers to remove cannot be null or empty.
	at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.removeServers(RSGroupAdminServer.java:391)
	at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.postClearDeadServers(RSGroupAdminEndpoint.java:1175)
	at org.apache.hadoop.hbase.master.MasterCoprocessorHost$104.call(MasterCoprocessorHost.java:1251)
	at org.apache.hadoop.hbase.master.MasterCoprocessorHost.execOperation(MasterCoprocessorHost.java:1507)
	at org.apache.hadoop.hbase.master.MasterCoprocessorHost.postClearDeadServers(MasterCoprocessorHost.java:1247)
	at org.apache.hadoop.hbase.master.MasterRpcServices.clearDeadServers(MasterRpcServices.java:1167)
	at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2421)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:311)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:291)

	at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.testClearDeadServers(TestRSGroupsBasics.java:215)
Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException: 
org.apache.hadoop.hbase.constraint.ConstraintException: The set of servers to remove cannot be null or empty.
	at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.removeServers(RSGroupAdminServer.java:391)
	at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.postClearDeadServers(RSGroupAdminEndpoint.java:1175)
	at org.apache.hadoop.hbase.master.MasterCoprocessorHost$104.call(MasterCoprocessorHost.java:1251)
	at org.apache.hadoop.hbase.master.MasterCoprocessorHost.execOperation(MasterCoprocessorHost.java:1507)
	at org.apache.hadoop.hbase.master.MasterCoprocessorHost.postClearDeadServers(MasterCoprocessorHost.java:1247)
	at org.apache.hadoop.hbase.master.MasterRpcServices.clearDeadServers(MasterRpcServices.java:1167)
	at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2421)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:311)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:291)

Issue Links

• relates to: HBASE-20927 RSGroupAdminEndpoint doesn't handle clearing dead servers if they are not processed yet.
• links to: GitHub Pull Request #1821