Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-1857

Ability to run a script when a task fails to capture stack traces

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.14.0
    • 0.16.0
    • None
    • None

    Description

      This basically is for providing a better user interface for debugging failed
      jobs. Today we see stack traces for failed tasks on the job ui if the job
      happened to be a Java MR job. For non-Java jobs like Streaming, Pipes, the
      diagnostic info on the job UI is not helpful enough to debug what might have
      gone wrong. They are usually framework traces and not app traces.
      We want to be able to provide a facility, via user-provided scripts, for doing
      post-processing on task logs, input, output, etc. There should be some default
      scripts like running core dumps under gdb for locating illegal instructions,
      the last few lines from stderr, etc. These outputs could be sent to the
      tasktracker and in turn to the jobtracker which would then display it on the
      job UI on demand.

      Attachments

        1. patch-1857.txt
          39 kB
          Amareshwari Sriramadasu
        2. patch-1857.txt
          39 kB
          Amareshwari Sriramadasu
        3. patch-1857.txt
          36 kB
          Amareshwari Sriramadasu
        4. patch-1857.txt
          36 kB
          Amareshwari Sriramadasu
        5. patch-1857.txt
          36 kB
          Amareshwari Sriramadasu
        6. patch-1857.txt
          36 kB
          Amareshwari Sriramadasu
        7. patch-1857.txt
          37 kB
          Amareshwari Sriramadasu
        8. patch-1857.txt
          53 kB
          Amareshwari Sriramadasu
        9. patch-1857.txt
          53 kB
          Amareshwari Sriramadasu
        10. patch-1857.txt
          53 kB
          Amareshwari Sriramadasu
        11. patch-1857.txt
          53 kB
          Amareshwari Sriramadasu
        12. tt-no-warn.patch
          11 kB
          Owen O'Malley

        Activity

          People

            amareshwari Amareshwari Sriramadasu
            amareshwari Amareshwari Sriramadasu
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: