VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.21.0
    • None
    • None
    • Reviewed
    • Removed the Job Retire thread and the associated configuration parameters. Job is purged from memory as soon as the history file is copied to HDFS. Only JobStatus object is retained in the retired jobs cache.

    Description

      Currently completed job's full data structures are kept in memory based on mapred.jobtracker.completeuserjobs.maximum, mapred.jobtracker.retirejob.interval.min, mapred.jobtracker.retirejob.interval and mapred.jobtracker.retirejob.check settings. These controls are not much useful now since MAPREDUCE-817 introduced a cache for keeping just the very basic info of the completed job. These settings should be removed and the job should be purged as soon as the history files are available in HDFS.
      Going forward, clients can read the history files if they need to drill down into more information (MAPREDUCE-864).

      Attachments

        1. 870_v1.patch
          27 kB
          Sharad Agarwal
        2. 870_v2.patch
          28 kB
          Sharad Agarwal
        3. 870_v3.patch
          30 kB
          Sharad Agarwal

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            sharadag Sharad Agarwal
            sharadag Sharad Agarwal
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment