[TEZ-796] AM Hangs & does not kill containers when map-task fails

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.3.0
• Fix Version/s: 0.3.0
• Assignee: Bikas Saha
• Reporter: Gopal Vijayaraghavan
• Created: 04/Feb/14 22:47
• Updated: 01/Mar/14 04:19
• Resolved: 12/Feb/14 06:32

Description

The task hangs after a vertex fails with an error and continues to idle without returning an error.

Looks like reducer spinups continue even after a vertex fails.

ask_1389745080241_1218_6_04_000974,KILL_WAIT,KILLED
task_1389745080241_1218_6_04_000976,KILL_WAIT,KILLED
task_1389745080241_1218_6_09_000270,SCHEDULED,RUNNING
task_1389745080241_1218_6_04_000978,KILL_WAIT,KILLED
task_1389745080241_1218_6_04_000979,KILL_WAIT,KILLED
task_1389745080241_1218_6_04_000980,KILL_WAIT,KILLED
task_1389745080241_1218_6_04_000981,KILL_WAIT,KILLED
task_1389745080241_1218_6_04_000982,KILL_WAIT,KILLED
task_1389745080241_1218_6_04_000984,KILL_WAIT,KILLED
task_1389745080241_1218_6_04_000987,KILL_WAIT,KILLED

In the attached log file, the map vertex fails at 10:59 and DAG_FINISHED is not triggered till 11:03, at which point the AM was killed by hand.