Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3973 [Umbrella JIRA] JobHistoryServer performance improvements in YARN+MR
  3. MAPREDUCE-3755

Add the equivalent of JobStatus to end of JobHistory file

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • 0.23.0
    • 0.23.2
    • jobhistoryserver, mrv2
    • None

    Description

      In MR1 we have the notion of CompletedJobStatus store to aid fast responses to job.getStatus. We need the equivalent for MR2, an option is to add the jobStatus to the end of the JobHistory file to which the JHS can easily jump ahead to and serve the query, it should also cache this for a fair number of recently completed jobs.

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-4442 Accessing hadoop counters from a job is unreliable in yarn during AM process cleanup window

          • Major - Major loss of function.
          • Open

          Activity

            People

              bikassaha Bikas Saha
              acmurthy Arun Murthy
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: