Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3901

lazy load JobHistory Task and TaskAttempt details

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.2
    • Component/s: jobhistoryserver, mrv2
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Modified JobHistory records in YARN to lazily load job and task reports so as to improve UI response times.
    • Target Version/s:

      Description

      The job history UI and MRClientProtocol calls routed via JobHistory are very slow for large jobs. Some of this time is spent parsing the history file. A good chunk is spent pre-creating lots of objects which may never be used. Those can be create when required - bringing down the load times of job history pages and getJobReport etc calls to approximately the history file parse time.

      1. MR3901.txt
        169 kB
        Siddharth Seth
      2. MR3901_v3.txt
        169 kB
        Siddharth Seth
      3. MR3901_v2.txt
        169 kB
        Siddharth Seth

        Activity

          People

          • Assignee:
            Siddharth Seth
            Reporter:
            Siddharth Seth
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development