Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-863 Improve/standardize job history file content format and the management of the history files
  3. MAPREDUCE-864

Enhance JobClient API implementations to look at history files to get information about jobs that are not in memory

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • jobtracker
    • None

    Description

      MAPREDUCE-817 added an API to get the JobHistory URL from the JobTracker. This is useful in two ways:
      1) Users can use this API to get the URL, copy the history files to their local disk, and, do processing on them
      2) APIs like JobSubmissionProtocol.getJobCounters, can read a part of the history file, and then return the information to the caller (if the job is not there in JT memory). This would mimic most of the CompletedJobsStatusStore functionality.

      Attachments

        Issue Links

          is blocked by

          Sub-task - The sub-task of the issue MAPREDUCE-157 Job History log file format is not friendly for external tools.

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. MAPREDUCE-777 A method for finding and tracking jobs from the new API

          • Major - Major loss of function.
          • Closed

          Activity

            People

              sharadag Sharad Agarwal
              ddas Devaraj Das
              Votes:
              1 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: