[HDDS-11358] Intermittent RATIS_BOOTSTRAP_ERROR in TestAddRemoveOzoneManager#testForceBootstrap

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Ethan Rose
• Created: 22/Aug/24 15:21
• Updated: 10/Sep/24 08:34

Description

Seen in this run

-------------------------------------------------------------------------------
Test set: org.apache.hadoop.ozone.om.TestAddRemoveOzoneManager
-------------------------------------------------------------------------------
Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 186.682 s <<< FAILURE! - in org.apache.hadoop.ozone.om.TestAddRemoveOzoneManager
org.apache.hadoop.ozone.om.TestAddRemoveOzoneManager.testForceBootstrap  Time elapsed: 58.511 s  <<< ERROR!
java.io.IOException: Failed to Bootstrap OM. Error Code: RATIS_BOOTSTRAP_ERROR, Error Message: omNode-1@group-0AAC5367B30E-ConfigurationStagingState: Fail to set configuration peers:[omNode-bootstrap-2|localhost:15058, omNode-3|localhost:15023, omNode-1|localhost:15015, omNode-2|localhost:15019]|listeners:[] due to NOPROGRESS
	at org.apache.hadoop.ozone.om.protocolPB.OMInterServiceProtocolClientSideImpl.throwException(OMInterServiceProtocolClientSideImpl.java:113)
	at org.apache.hadoop.ozone.om.protocolPB.OMInterServiceProtocolClientSideImpl.bootstrap(OMInterServiceProtocolClientSideImpl.java:107)
	at org.apache.hadoop.ozone.om.OzoneManager.bootstrap(OzoneManager.java:1935)
	at org.apache.hadoop.ozone.om.OzoneManager.start(OzoneManager.java:1743)
	at org.apache.hadoop.ozone.MiniOzoneHAClusterImpl$MiniOzoneHAService.startInactiveService(MiniOzoneHAClusterImpl.java:993)
	at org.apache.hadoop.ozone.MiniOzoneHAClusterImpl.startInactiveOM(MiniOzoneHAClusterImpl.java:189)
	at org.apache.hadoop.ozone.MiniOzoneHAClusterImpl.bootstrapNewOM(MiniOzoneHAClusterImpl.java:834)
	at org.apache.hadoop.ozone.MiniOzoneHAClusterImpl.bootstrapOzoneManager(MiniOzoneHAClusterImpl.java:749)
	at org.apache.hadoop.ozone.om.TestAddRemoveOzoneManager.testForceBootstrap(TestAddRemoveOzoneManager.java:338)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

Issue Links

• is caused by: RATIS-2152 GrpcLogAppender stucks while sending an installSnapshot notification request
• relates to: HDDS-7880 Intermittent timeout in TestAddRemoveOzoneManager.testBootstrap