[HDDS-6830] EC: SCMContainerPlacementRackScatter#chooseDatanodes may choose less nodes than required in unknown cases.

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.3.0
• Labels: pull-request-available
• Assignee: Attila Doroszlai
• Reporter: Nilotpal Nandi
• Created: 03/Jun/22 18:16
• Updated: 14/Jun/22 21:37
• Resolved: 14/Jun/22 16:51

Description

This issue may be shaded because we have a sanity check at the end of chooseDatanodes method and throw SCM Exception. However, it fail in the above checks itself if it cannot choose enough nodes. With the sanity check the following condition we may not meet, but we can still investigate why SCMContainerPlacementRackScatter giving up silently without exception. 

2022-05-19 20:15:35,536 ERROR org.apache.ratis.statemachine.StateMachine: Terminating with exit status 1: Nodes size=4, replication factor=5 do not match java.lang.IllegalArgumentException: Nodes size=4, replication factor=5 do not match at com.google.common.base.Preconditions.checkArgument(Preconditions.java:303) at org.apache.hadoop.hdds.scm.pipeline.PipelineStateMap.addPipeline(PipelineStateMap.java:74) at org.apache.hadoop.hdds.scm.pipeline.PipelineStateManagerImpl.addPipeline(PipelineStateManagerImpl.java:99) at jdk.internal.reflect.GeneratedMethodAccessor222.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.hadoop.hdds.scm.ha.SCMStateMachine.process(SCMStateMachine.java:168) at org.apache.hadoop.hdds.scm.ha.SCMStateMachine.applyTransaction(SCMStateMachine.java:139) at org.apache.ratis.server.impl.RaftServerImpl.applyLogToStateMachine(RaftServerImpl.java:1723) at org.apache.ratis.server.impl.StateMachineUpdater.applyLog(StateMachineUpdater.java:234) at org.apache.ratis.server.impl.StateMachineUpdater.run(StateMachineUpdater.java:179) at java.base/java.lang.Thread.run(Thread.java:834)

Issue Links

• links to: GitHub Pull Request #3511