Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5641

History for failed Application Masters should be made available to the Job History Server

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.2.0
    • Fix Version/s: None
    • Labels:
      None

      Description

      Currently, the JHS has no information about jobs whose AMs have failed. This is because the History is written by the AM to the intermediate folder just before finishing, so when it fails for any reason, this information isn't copied there. However, it is not lost as its in the AM's staging directory. To make the History available in the JHS, all we need to do is have another mechanism to move the History from the staging directory to the intermediate directory. The AM also writes a "Summary" file before exiting normally, which is also unavailable when the AM fails.

        Attachments

        1. MAPREDUCE-5641.patch
          27 kB
          Robert Kanter
        2. MAPREDUCE-5641.patch
          27 kB
          Robert Kanter

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                rkanter Robert Kanter
              • Votes:
                1 Vote for this issue
                Watchers:
                18 Start watching this issue

                Dates

                • Created:
                  Updated: