[FLINK-34272] AdaptiveSchedulerClusterITCase failure due to MiniCluster not running

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.19.0
• Fix Version/s: 1.19.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available test-stability
• Assignee: David Morávek
• Reporter: Matthias Pohl
• Created: 30/Jan/24 07:54
• Updated: 06/Feb/24 10:18
• Resolved: 01/Feb/24 02:09

Description

https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=57073&view=logs&j=0da23115-68bb-5dcd-192c-bd4c8adebde1&t=24c3384f-1bcb-57b3-224f-51bf973bbee8&l=9543

 Jan 29 17:21:29 17:21:29.465 [ERROR] Tests run: 3, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 12.48 s <<< FAILURE! -- in org.apache.flink.runtime.scheduler.adaptive.AdaptiveSchedulerClusterITCase
Jan 29 17:21:29 17:21:29.465 [ERROR] org.apache.flink.runtime.scheduler.adaptive.AdaptiveSchedulerClusterITCase.testAutomaticScaleUp -- Time elapsed: 8.599 s <<< ERROR!
Jan 29 17:21:29 java.lang.IllegalStateException: MiniCluster is not yet running or has already been shut down.
Jan 29 17:21:29 	at org.apache.flink.util.Preconditions.checkState(Preconditions.java:193)
Jan 29 17:21:29 	at org.apache.flink.runtime.minicluster.MiniCluster.getDispatcherGatewayFuture(MiniCluster.java:1118)
Jan 29 17:21:29 	at org.apache.flink.runtime.minicluster.MiniCluster.runDispatcherCommand(MiniCluster.java:991)
Jan 29 17:21:29 	at org.apache.flink.runtime.minicluster.MiniCluster.getArchivedExecutionGraph(MiniCluster.java:840)
Jan 29 17:21:29 	at org.apache.flink.runtime.scheduler.adaptive.AdaptiveSchedulerClusterITCase.lambda$waitUntilParallelismForVertexReached$3(AdaptiveSchedulerClusterITCase.java:270)
Jan 29 17:21:29 	at org.apache.flink.runtime.testutils.CommonTestUtils.waitUntilCondition(CommonTestUtils.java:151)
Jan 29 17:21:29 	at org.apache.flink.runtime.testutils.CommonTestUtils.waitUntilCondition(CommonTestUtils.java:145)
Jan 29 17:21:29 	at org.apache.flink.runtime.scheduler.adaptive.AdaptiveSchedulerClusterITCase.waitUntilParallelismForVertexReached(AdaptiveSchedulerClusterITCase.java:265)
Jan 29 17:21:29 	at org.apache.flink.runtime.scheduler.adaptive.AdaptiveSchedulerClusterITCase.testAutomaticScaleUp(AdaptiveSchedulerClusterITCase.java:146)
Jan 29 17:21:29 	at java.lang.reflect.Method.invoke(Method.java:498)
Jan 29 17:21:29 	at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
Jan 29 17:21:29 
Jan 29 17:21:29 17:21:29.466 [ERROR] org.apache.flink.runtime.scheduler.adaptive.AdaptiveSchedulerClusterITCase.testCheckpointStatsPersistedAcrossRescale -- Time elapsed: 2.036 s <<< ERROR!
Jan 29 17:21:29 java.lang.IllegalStateException: MiniCluster is not yet running or has already been shut down.
Jan 29 17:21:29 	at org.apache.flink.util.Preconditions.checkState(Preconditions.java:193)
Jan 29 17:21:29 	at org.apache.flink.runtime.minicluster.MiniCluster.getDispatcherGatewayFuture(MiniCluster.java:1118)
Jan 29 17:21:29 	at org.apache.flink.runtime.minicluster.MiniCluster.runDispatcherCommand(MiniCluster.java:991)
Jan 29 17:21:29 	at org.apache.flink.runtime.minicluster.MiniCluster.getExecutionGraph(MiniCluster.java:969)
Jan 29 17:21:29 	at org.apache.flink.runtime.scheduler.adaptive.AdaptiveSchedulerClusterITCase.lambda$testCheckpointStatsPersistedAcrossRescale$1(AdaptiveSchedulerClusterITCase.java:183)
Jan 29 17:21:29 	at org.apache.flink.runtime.testutils.CommonTestUtils.waitUntilCondition(CommonTestUtils.java:151)
Jan 29 17:21:29 	at org.apache.flink.runtime.testutils.CommonTestUtils.waitUntilCondition(CommonTestUtils.java:145)
Jan 29 17:21:29 	at org.apache.flink.runtime.scheduler.adaptive.AdaptiveSchedulerClusterITCase.testCheckpointStatsPersistedAcrossRescale(AdaptiveSchedulerClusterITCase.java:180)
Jan 29 17:21:29 	at java.lang.reflect.Method.invoke(Method.java:498)
Jan 29 17:21:29 	at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)

Issue Links

• is caused by: FLINK-21883 Introduce cooldown period into adaptive scheduler
• is related to: FLINK-34274 AdaptiveSchedulerTest.testRequirementLowerBoundDecreaseAfterResourceScarcityBelowAvailableSlots times out
• links to: GitHub Pull Request #24236
• links to: GitHub Pull Request #24237