[FLINK-16136] YarnPrioritySchedulingITCase.yarnApplication_submissionWithPriority_shouldRespectPriority() fails

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.11.0
• Fix Version/s: 1.11.0
• Component/s: Deployment / YARN, Tests
• Labels: pull-request-available test-stability
• Assignee: Robert Metzger
• Reporter: Robert Metzger
• Created: 18/Feb/20 09:52
• Updated: 27/Apr/21 18:06
• Resolved: 18/Feb/20 14:37

Description

https://dev.azure.com/rmetzger/Flink/_build/results?buildId=5259&view=logs&j=764762df-f65b-572b-3d5c-65518c777be4&t=da3c2718-4b76-56bf-ef25-cd33ea381f78
https://transfer.sh/G229r/20200218.1.tar.gz

01:29:45,184 [                main] INFO  org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl  - Notifying ContainerManager to unblock new container-requests
01:29:45,184 [ResourceManager Event Processor] INFO  org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler  - Added node 8b508210238a:37199 clusterResource: <memory:4096, vCores:666>
01:29:45,192 [                main] INFO  org.apache.hadoop.yarn.server.MiniYARNCluster                 - All Node Managers connected in MiniYARNCluster
01:29:45,192 [AsyncDispatcher event handler] INFO  org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl  - Node 8b508210238a:37199 reported UNHEALTHY with details: 1/1 local-dirs are bad: /__w/1/s/flink-yarn-tests/target/YarnTest_3d15de1b-4c9a-411b-81b1-3ca2788b479a/YarnTest_3d15de1b-4c9a-411b-81b1-3ca2788b479a-localDir-nm-1_0; 1/1 log-dirs are bad: /__w/1/s/flink-yarn-tests/target/YarnTest_3d15de1b-4c9a-411b-81b1-3ca2788b479a/YarnTest_3d15de1b-4c9a-411b-81b1-3ca2788b479a-logDir-nm-1_0
01:29:45,192 [AsyncDispatcher event handler] INFO  org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl  - 8b508210238a:37199 Node Transitioned from RUNNING to UNHEALTHY
01:29:45,193 [ResourceManager Event Processor] INFO  org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler  - Removed node 8b508210238a:37199 clusterResource: <memory:0, vCores:0>

================================================================================
Test yarnApplication_submissionWithPriority_shouldRespectPriority(org.apache.flink.yarn.YarnPrioritySchedulingITCase) is running.
--------------------------------------------------------------------------------
01:29:45,449 [                main] INFO  org.apache.hadoop.yarn.client.RMProxy                         - Connecting to ResourceManager at 8b508210238a/172.18.0.2:40977
01:29:45,529 [                main] INFO  org.apache.flink.yarn.YarnTestBase                            - Running with args [-j, /__w/1/s/flink-yarn-tests/../build-target/lib/flink-dist_2.11-1.11-SNAPSHOT.jar, -t, /__w/1/s/flink-yarn-tests/../build-target/lib, -t, /__w/1/s/flink-yarn-tests/target/shaded-hadoop, -jm, 768m, -tm, 1024m, -Dyarn.application.priority=5]
01:29:45,555 [Frontend (CLI/YARN Client) runner thread (startWithArgs()).] WARN  org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - The configuration directory ('/tmp/junit3457413859979027256/conf') already contains a LOG4J config file.If you want to use logback, then please delete or rename the log configuration file.
01:29:45,567 [Frontend (CLI/YARN Client) runner thread (startWithArgs()).] INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - Dynamic Property set: yarn.application.priority=5
01:29:45,591 [Frontend (CLI/YARN Client) runner thread (startWithArgs()).] INFO  org.apache.hadoop.yarn.client.RMProxy                         - Connecting to ResourceManager at 8b508210238a/172.18.0.2:40977
01:29:45,611 [Frontend (CLI/YARN Client) runner thread (startWithArgs()).] INFO  org.apache.flink.runtime.clusterframework.TaskExecutorProcessUtils  - The derived from fraction jvm overhead memory (102.400mb (107374184 bytes)) is less than its min value 192.000mb (201326592 bytes), min value will be used instead
01:29:45,753 [Frontend (CLI/YARN Client) runner thread (startWithArgs()).] INFO  org.apache.flink.yarn.YarnTestBase                            - Runner stopped with exception
org.apache.flink.client.deployment.ClusterDeploymentException: Couldn't deploy Yarn session cluster
	at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:381)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:548)
	at org.apache.flink.yarn.YarnTestBase$Runner.run(YarnTestBase.java:917)
Caused by: org.apache.flink.configuration.IllegalConfigurationException: The number of requested virtual cores for application master 1 exceeds the maximum number of virtual cores 0 available in the Yarn Cluster.
	at org.apache.flink.yarn.YarnClusterDescriptor.isReadyForDeployment(YarnClusterDescriptor.java:284)
	at org.apache.flink.yarn.YarnClusterDescriptor.deployInternal(YarnClusterDescriptor.java:444)
	at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:374)
	... 2 more

Issue Links

• links to: GitHub Pull Request #11122