[HDDS-5233] SCM subsequent init failed when previous scm init failed

• Type: Sub-task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.2.0
• Component/s: SCM HA
• Labels: pull-request-available
• Assignee: Bharat Viswanadham
• Reporter: George Huang
• Created: 17/May/21 08:45
• Updated: 21/May/21 04:00
• Resolved: 21/May/21 04:00

Description

The problem is SCM init because we use a new clusterID when the version writing failed.

Could not initialize SCM version file
java.io.IOException: java.lang.IllegalStateException: ILLEGAL TRANSITION: In SCMStateMachine:eca7c0f7-9310-45a4-bee2-76a3242dd372:group-010A6AE5EDB4, RUNNING -> STARTING
	at org.apache.ratis.util.IOUtils.asIOException(IOUtils.java:54)
	at org.apache.ratis.util.IOUtils.toIOException(IOUtils.java:61)
	at org.apache.ratis.util.IOUtils.getFromFuture(IOUtils.java:71)
	at org.apache.ratis.server.impl.RaftServerProxy.getImpls(RaftServerProxy.java:354)
	at org.apache.ratis.server.impl.RaftServerProxy.start(RaftServerProxy.java:371)
	at org.apache.hadoop.hdds.scm.ha.SCMRatisServerImpl.initialize(SCMRatisServerImpl.java:115)
	at org.apache.hadoop.hdds.scm.server.StorageContainerManager.scmInit(StorageContainerManager.java:925)
	at org.apache.hadoop.hdds.scm.server.StorageContainerManagerStarter$SCMStarterHelper.init(StorageContainerManagerStarter.java:173)
	at org.apache.hadoop.hdds.scm.server.StorageContainerManagerStarter.initScm(StorageContainerManagerStarter.java:110)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1952)
	at picocli.CommandLine.access$1100(CommandLine.java:145)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2332)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2326)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2291)
	at picocli.CommandLine$AbstractParseResultHandler.handleParseResult(CommandLine.java:2152)
	at picocli.CommandLine.parseWithHandlers(CommandLine.java:2530)
	at picocli.CommandLine.parseWithHandler(CommandLine.java:2465)
	at org.apache.hadoop.hdds.cli.GenericCli.execute(GenericCli.java:96)
	at org.apache.hadoop.hdds.cli.GenericCli.run(GenericCli.java:87)
	at org.apache.hadoop.hdds.scm.server.StorageContainerManagerStarter.main(StorageContainerManagerStarter.java:57)
Caused by: java.lang.IllegalStateException: ILLEGAL TRANSITION: In SCMStateMachine:eca7c0f7-9310-45a4-bee2-76a3242dd372:group-010A6AE5EDB4, RUNNING -> STARTING

Issue Links

• links to: GitHub Pull Request #2249