[TEZ-878] doAssignAll() in TaskScheduler ignores delayedContainers being out of sync with heldContainers

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.3.0
• Fix Version/s: 0.3.0
• Assignee: Bikas Saha
• Reporter: Tassapol Athiapinya
• Created: 22/Feb/14 00:32
• Updated: 01/Mar/14 04:19
• Resolved: 22/Feb/14 20:32

Description

I have single-node cluster. I run tez DAG similar to testInputFailureCausesRerunOfTwoVerticesWithoutExit unit test added in TEZ-823. DAG does not exit. There is an NPE in Tez AM logs:

2014-02-21 23:42:58,123 INFO [AsyncDispatcher event handler] org.apache.tez.dag.app.dag.impl.TaskImpl: task_1393025990757_0002_1_00_000000 Task Transitioned from SUCCEEDED to SCHEDULED
2014-02-21 23:42:58,124 ERROR [DelayedContainerManager] org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread Thread[DelayedContainerManager,5,main] threw an Exception.
java.lang.NullPointerException
	at org.apache.tez.dag.app.rm.TaskScheduler.assignContainer(TaskScheduler.java:1084)
	at org.apache.tez.dag.app.rm.TaskScheduler.access$500(TaskScheduler.java:83)
	at org.apache.tez.dag.app.rm.TaskScheduler$ContainerAssigner.doBookKeepingForAssignedContainer(TaskScheduler.java:1326)
	at org.apache.tez.dag.app.rm.TaskScheduler$NodeLocalContainerAssigner.assignReUsedContainer(TaskScheduler.java:1351)
	at org.apache.tez.dag.app.rm.TaskScheduler.assignReUsedContainerWithLocation(TaskScheduler.java:1231)
	at org.apache.tez.dag.app.rm.TaskScheduler.assignReUsedContainersWithLocation(TaskScheduler.java:1193)
	at org.apache.tez.dag.app.rm.TaskScheduler.tryAssignReUsedContainers(TaskScheduler.java:505)
	at org.apache.tez.dag.app.rm.TaskScheduler.access$1000(TaskScheduler.java:83)
	at org.apache.tez.dag.app.rm.TaskScheduler$DelayedContainerManager.doAssignAll(TaskScheduler.java:1554)
	at org.apache.tez.dag.app.rm.TaskScheduler$DelayedContainerManager.run(TaskScheduler.java:1456)