[HDDS-2392] Fix TestScmSafeMode#testSCMSafeModeRestrictedOp

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Target Version/s: 0.5.0
• Assignee: Hanisha Koneru
• Reporter: Hanisha Koneru
• Created: 31/Oct/19 19:55
• Updated: 13/Nov/19 21:30
• Resolved: 13/Nov/19 21:29

Description

After ratis upgrade (HDDS-2340), TestScmSafeMode#testSCMSafeModeRestrictedOp fails as the DNs fail to restart XceiverServerRatis. 

RaftServer#start() fails with following exception:

java.io.IOException: java.lang.IllegalStateException: Not started
	at org.apache.ratis.util.IOUtils.asIOException(IOUtils.java:54)
	at org.apache.ratis.util.IOUtils.toIOException(IOUtils.java:61)
	at org.apache.ratis.util.IOUtils.getFromFuture(IOUtils.java:70)
	at org.apache.ratis.server.impl.RaftServerProxy.getImpls(RaftServerProxy.java:284)
	at org.apache.ratis.server.impl.RaftServerProxy.start(RaftServerProxy.java:296)
	at org.apache.hadoop.ozone.container.common.transport.server.ratis.XceiverServerRatis.start(XceiverServerRatis.java:421)
	at org.apache.hadoop.ozone.container.ozoneimpl.OzoneContainer.start(OzoneContainer.java:215)
	at org.apache.hadoop.ozone.container.common.states.endpoint.VersionEndpointTask.call(VersionEndpointTask.java:110)
	at org.apache.hadoop.ozone.container.common.states.endpoint.VersionEndpointTask.call(VersionEndpointTask.java:42)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Not started
	at org.apache.ratis.thirdparty.com.google.common.base.Preconditions.checkState(Preconditions.java:504)
	at org.apache.ratis.thirdparty.io.grpc.internal.ServerImpl.getPort(ServerImpl.java:176)
	at org.apache.ratis.grpc.server.GrpcService.lambda$new$2(GrpcService.java:143)
	at org.apache.ratis.util.MemoizedSupplier.get(MemoizedSupplier.java:62)
	at org.apache.ratis.grpc.server.GrpcService.getInetSocketAddress(GrpcService.java:182)
	at org.apache.ratis.server.impl.RaftServerImpl.lambda$new$0(RaftServerImpl.java:84)
	at org.apache.ratis.util.MemoizedSupplier.get(MemoizedSupplier.java:62)
	at org.apache.ratis.server.impl.RaftServerImpl.getPeer(RaftServerImpl.java:136)
	at org.apache.ratis.server.impl.RaftServerMetrics.<init>(RaftServerMetrics.java:70)
	at org.apache.ratis.server.impl.RaftServerMetrics.getRaftServerMetrics(RaftServerMetrics.java:62)
	at org.apache.ratis.server.impl.RaftServerImpl.<init>(RaftServerImpl.java:119)
	at org.apache.ratis.server.impl.RaftServerProxy.lambda$newRaftServerImpl$2(RaftServerProxy.java:208)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)

Issue Links

• duplicates: HDDS-2367 TestScmSafeMode fails consistently
• is related to: RATIS-747 RaftServerMetrics.getRaftServerMetrics throws IllegalStateException