[HDDS-9805] Reduce cluster size in integration tests

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.0
• Component/s: test
• Labels: pull-request-available
• Assignee: Arafat Khan
• Reporter: Tsz-wo Sze
• Created: 30/Nov/23 07:04
• Updated: 07/Jan/24 18:44
• Resolved: 07/Dec/23 17:24

Description

The following tests seem to create too many datenode and pipelines. How about change it to 5 datanodes and 3 pipelines?

TestOzoneClientRetriesOnExceptions.java-112-    cluster = MiniOzoneCluster.newBuilder(conf)
TestOzoneClientRetriesOnExceptions.java:113:        .setNumDatanodes(7)
TestOzoneClientRetriesOnExceptions.java-114-        .setTotalPipelineNumLimit(10)
--
TestBlockOutputStream.java-97-    cluster = MiniOzoneCluster.newBuilder(conf)
TestBlockOutputStream.java:98:        .setNumDatanodes(7)
TestBlockOutputStream.java-99-        .setTotalPipelineNumLimit(10)
--
TestCloseContainerHandlingByClient.java-106-        StorageUnit.MB);
TestCloseContainerHandlingByClient.java:107:    cluster = MiniOzoneCluster.newBuilder(conf).setNumDatanodes(7).build();
TestCloseContainerHandlingByClient.java-108-    cluster.waitForClusterToBeReady();
--
TestBlockOutputStreamFlushDelay.java-89-    cluster = MiniOzoneCluster.newBuilder(conf)
TestBlockOutputStreamFlushDelay.java:90:        .setNumDatanodes(7)
TestBlockOutputStreamFlushDelay.java-91-        .setTotalPipelineNumLimit(10)
--
TestBlockDataStreamOutput.java-101-    cluster = MiniOzoneCluster.newBuilder(conf)
TestBlockDataStreamOutput.java:102:        .setNumDatanodes(7)
TestBlockDataStreamOutput.java-103-        .setTotalPipelineNumLimit(10)
--
TestOzoneClientRetriesOnExceptionFlushDelay.java-104-    cluster = MiniOzoneCluster.newBuilder(conf)
TestOzoneClientRetriesOnExceptionFlushDelay.java:105:        .setNumDatanodes(7)
TestOzoneClientRetriesOnExceptionFlushDelay.java-106-        .setTotalPipelineNumLimit(10)
--
TestBlockOutputStreamWithFailures.java-124-
TestBlockOutputStreamWithFailures.java:125:    cluster = MiniOzoneCluster.newBuilder(conf).setNumDatanodes(7)
TestBlockOutputStreamWithFailures.java-126-        .setTotalPipelineNumLimit(10).setBlockSize(blockSize)
--
TestBlockOutputStreamWithFailuresFlushDelay.java-126-
TestBlockOutputStreamWithFailuresFlushDelay.java:127:    cluster = MiniOzoneCluster.newBuilder(conf).setNumDatanodes(7)
TestBlockOutputStreamWithFailuresFlushDelay.java-128-        .setTotalPipelineNumLimit(10).setBlockSize(blockSize)
--
TestCommitWatcher.java-143-    cluster = MiniOzoneCluster.newBuilder(conf)
TestCommitWatcher.java:144:        .setNumDatanodes(7)
TestCommitWatcher.java-145-        .setTotalPipelineNumLimit(10)
--
TestOzoneFsHAURLs.java-126-    cluster = MiniOzoneCluster.newOMHABuilder(conf)
TestOzoneFsHAURLs.java:127:        .setNumDatanodes(7)
TestOzoneFsHAURLs.java-128-        .setTotalPipelineNumLimit(10)

Issue Links

• Discovered while testing: HDDS-9766 Intermittent AlreadyClosedException in TestCommitWatcher.testReleaseBuffersOnException
• links to: GitHub Pull Request #5706