Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3901

lazy load JobHistory Task and TaskAttempt details

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.2
    • Component/s: jobhistoryserver, mrv2
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Modified JobHistory records in YARN to lazily load job and task reports so as to improve UI response times.

      Description

      The job history UI and MRClientProtocol calls routed via JobHistory are very slow for large jobs. Some of this time is spent parsing the history file. A good chunk is spent pre-creating lots of objects which may never be used. Those can be create when required - bringing down the load times of job history pages and getJobReport etc calls to approximately the history file parse time.

        Attachments

        1. MR3901_v2.txt
          169 kB
          Siddharth Seth
        2. MR3901_v3.txt
          169 kB
          Siddharth Seth
        3. MR3901.txt
          169 kB
          Siddharth Seth

          Activity

            People

            • Assignee:
              sseth Siddharth Seth
              Reporter:
              sseth Siddharth Seth
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: