[HADOOP-1857] Ability to run a script when a task fails to capture stack traces

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.14.0
• Fix Version/s: 0.16.0
• Assignee: Amareshwari Sriramadasu
• Reporter: Amareshwari Sriramadasu
• Created: 07/Sep/07 08:57
• Updated: 08/Jul/09 16:52
• Resolved: 26/Oct/07 09:43

Description

This basically is for providing a better user interface for debugging failed
jobs. Today we see stack traces for failed tasks on the job ui if the job
happened to be a Java MR job. For non-Java jobs like Streaming, Pipes, the
diagnostic info on the job UI is not helpful enough to debug what might have
gone wrong. They are usually framework traces and not app traces.
We want to be able to provide a facility, via user-provided scripts, for doing
post-processing on task logs, input, output, etc. There should be some default
scripts like running core dumps under gdb for locating illegal instructions,
the last few lines from stderr, etc. These outputs could be sent to the
tasktracker and in turn to the jobtracker which would then display it on the
job UI on demand.