[YARN-10297] TestContinuousScheduling#testFairSchedulerContinuousSchedulingInitTime fails intermittently

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.2, 2.10.1, 3.4.0, 3.3.1
• Fix Version/s: 3.2.2, 2.10.1, 3.4.0, 3.3.1
• Component/s: fairscheduler, test
• Target Version/s: 3.2.2, 2.10.1, 3.4.0, 3.3.1
• Hadoop Flags: Reviewed
• Assignee: Jim Brennan
• Reporter: Jonathan Hung
• Created: 29/May/20 21:25
• Updated: 11/Feb/24 06:33
• Resolved: 13/Jul/20 22:04

Description

After YARN-6492, testFairSchedulerContinuousSchedulingInitTime fails intermittently when running mvn test -Dtest=TestContinuousScheduling

[INFO] Running org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestContinuousScheduling
[ERROR] Tests run: 7, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 6.682 s <<< FAILURE! - in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestContinuousScheduling
[ERROR] testFairSchedulerContinuousSchedulingInitTime(org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestContinuousScheduling)  Time elapsed: 0.194 s  <<< ERROR!
org.apache.hadoop.metrics2.MetricsException: Metrics source PartitionQueueMetrics,partition= already exists!
	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newSourceName(DefaultMetricsSystem.java:152)
	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.sourceName(DefaultMetricsSystem.java:125)
	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:229)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.getPartitionMetrics(QueueMetrics.java:362)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.incrPendingResources(QueueMetrics.java:601)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.updatePendingResources(AppSchedulingInfo.java:388)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.internalAddResourceRequests(AppSchedulingInfo.java:320)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.internalAddResourceRequests(AppSchedulingInfo.java:347)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.updateResourceRequests(AppSchedulingInfo.java:183)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt.updateResourceRequests(SchedulerApplicationAttempt.java:456)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.allocate(FairScheduler.java:898)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestContinuousScheduling.testFairSchedulerContinuousSchedulingInitTime(TestContinuousScheduling.java:375)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)

Issue Links

• is broken by: YARN-6492 Generate queue metrics for each partition
• relates to: YARN-10329 Flaky test cases in Fair Scheduler