Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3901

lazy load JobHistory Task and TaskAttempt details

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.23.0
    • 0.23.2
    • jobhistoryserver, mrv2
    • None
    • Reviewed
    • Modified JobHistory records in YARN to lazily load job and task reports so as to improve UI response times.

    Description

      The job history UI and MRClientProtocol calls routed via JobHistory are very slow for large jobs. Some of this time is spent parsing the history file. A good chunk is spent pre-creating lots of objects which may never be used. Those can be create when required - bringing down the load times of job history pages and getJobReport etc calls to approximately the history file parse time.

      Attachments

        1. MR3901_v2.txt
          169 kB
          Siddharth Seth
        2. MR3901_v3.txt
          169 kB
          Siddharth Seth
        3. MR3901.txt
          169 kB
          Siddharth Seth

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            sseth Siddharth Seth
            sseth Siddharth Seth
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment