[YARN-9404] TestApplicationLifetimeMonitor#testApplicationLifetimeMonitor fails intermittent

• Type: Test
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.3.0, 2.10.1
• Component/s: resourcemanager
• Assignee: Prabhu Joseph
• Reporter: Prabhu Joseph
• Created: 22/Mar/19 15:18
• Updated: 12/May/20 13:57
• Resolved: 22/Mar/19 18:48

Description

TestApplicationLifetimeMonitor#testApplicationLifetimeMonitor fails intermittent.

[ERROR] testApplicationLifetimeMonitor[0](org.apache.hadoop.yarn.server.resourcemanager.rmapp.TestApplicationLifetimeMonitor) Time elapsed: 34.75 s <<< FAILURE! java.lang.AssertionError: Application killed before lifetime value at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at org.apache.hadoop.yarn.server.resourcemanager.rmapp.TestApplicationLifetimeMonitor.testApplicationLifetimeMonitor(TestApplicationLifetimeMonitor.java:209) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:748)

As per testcase logs, submittime is 1553240813597 and finishtime is 1553240844372. The testcase does (finishtime - submittime) / 1000 = 30775 / 1000 = 30 and loses the decimal, 775 ms.

2019-03-22 07:47:24,357 INFO  [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(149)) - Expired:application_1553240811329_0004_LIFETIME Timed out after 0 secs

2019-03-22 07:47:24,384 INFO  [AsyncDispatcher event handler] resourcemanager.RMAppManager$ApplicationSummary (RMAppManager.java:logAppSummary(219)) - appId=application_1553240811329_0004,name=,user=jenkins,queue=default,state=KILLED,trackingUrl=http://869e1f448cdd:8088/cluster/app/application_1553240811329_0004,appMasterHost=N/A,submitTime=1553240813597,startTime=1553240813604,launchTime=0,finishTime=1553240844372,finalStatus=KILLED,memorySeconds=0,vcoreSeconds=0,preemptedMemorySeconds=0,preemptedVcoreSeconds=0,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources=<memory:0\, vCores:0>,applicationType=YARN,resourceSeconds=0 MB-seconds\, 0 vcore-seconds,preemptedResourceSeconds=0 MB-seconds\, 0 vcore-seconds

Testcase succeeds only when the seconds taken is above 30L.

 long totalTimeRun =
            (app4.getFinishTime() - app4.getSubmitTime()) / 1000;
 Assert.assertTrue("Application killed before lifetime value",
            totalTimeRun > maxLifetime);

Issue Links

• is duplicated by: YARN-9436 Flaky test testApplicationLifetimeMonitor