[TEZ-3932] TaskSchedulerManager can throw NullPointerException during DAGAppMaster container cleanup race

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.10.0
• Fix Version/s: 0.9.2, 0.10.0
• Labels: ppc x86

Environment

arch: x86 and ppc

java: openjdk version "1.8.0_161"
         OpenJDK Runtime Environment (build 1.8.0_161-b14)
         OpenJDK 64-Bit Server VM (build 25.161-b14, mixed mode)

• Hadoop Flags: Reviewed
• Assignee: Jonathan Turner Eagles
• Reporter: Valencia Edna Serrao
• Created: 07/May/18 06:17
• Updated: 19/Apr/19 19:21
• Resolved: 09/May/18 15:31

Description

Test org.apache.tez.test.TestExceptionPropagation.testExceptionPropagationSession on x86 and ppc. I found related JIRA's TEZ-3746 and TEZ-3748. Though the issue is marked as resolved in the related JIRA's, the issue exists. Below are the error details:

-------------------------------------------------------------------------------
Test set: org.apache.tez.test.TestExceptionPropagation
-------------------------------------------------------------------------------
Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 96.433 sec <<< FAILURE!
testExceptionPropagationSession(org.apache.tez.test.TestExceptionPropagation)  Time elapsed: 52.7 sec  <<< ERROR!
org.apache.tez.dag.api.SessionNotRunning: Application not running, applicationId=application_1525667420557_0001, yarnApplicationState=FAILED, finalApplicationStatus=FAILED, trackingUrl=N/A, diagnostics=[DAG completed with an ERROR state. Shutting down AM, Session stats:submittedDAGs=11, successfulDAGs=0, failedDAGs=12, killedDAGs=0]
        at org.apache.tez.client.TezClientUtils.getAMProxy(TezClientUtils.java:910)
        at org.apache.tez.client.TezClient.getAMProxy(TezClient.java:1024)
        at org.apache.tez.client.TezClient.waitForProxy(TezClient.java:1034)
        at org.apache.tez.client.TezClient.submitDAGSession(TezClient.java:652)
        at org.apache.tez.client.TezClient.submitDAG(TezClient.java:588)
        at org.apache.tez.test.TestExceptionPropagation.testExceptionPropagationSession(TestExceptionPropagation.java:227