Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-449

There is little information provided when the TaskTracker kills a Task that has not reported within the timeout (600 sec) interval - this patch provides a stack trace of the task

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Minor
    • Resolution: Duplicate
    • None
    • None
    • None
    • None

    Description

      When we have a task that is killed for not reporting, sometimes there is an obvious programming error, and sometimes the reason the job didn't report is unclear.
      This patch will cause the TaskTracker to try to generate a stack trace of the offending task before the task is killed.
      Given how opaque process control is in java, a program is run to generate the stack trace, using the PID extracted from the undocumented UNIXProcess class

      The attached patch is against 0.16.0, as that is the release we use.
      This will only work on Unix machines – or JVM's what use the java.lang.UNIXProcess implementation for the java Process object.
      The script that generates the stack trace is very linux specific.
      The code changes will run on jvm's where the UNIXProcess class is not available, without failure, but no stack trace will be generated.

      Attachments

        1. 0.16_patch
          7 kB
          Jason

        Activity

          People

            Unassigned Unassigned
            jason_attributor Jason
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: