Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: client, jobtracker
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Hide
      Provides a way to configure the cache of JobStatus objects for the retired jobs.
      Adds an API in RunningJob to access history file url.
      Adds a LRU based cache for job history files loaded in memory when accessed via JobTracker web UI.
      Adds Retired Jobs table on the Jobtracker UI. The job move from Running to Completed/Failed table. Then job move to Retired table when it is purged from memory. The Retired table shows last 100 retired jobs. The Completed/Failed jobs table are only shown if there are non-zero jobs in the table.
      Show
      Provides a way to configure the cache of JobStatus objects for the retired jobs. Adds an API in RunningJob to access history file url. Adds a LRU based cache for job history files loaded in memory when accessed via JobTracker web UI. Adds Retired Jobs table on the Jobtracker UI. The job move from Running to Completed/Failed table. Then job move to Retired table when it is purged from memory. The Retired table shows last 100 retired jobs. The Completed/Failed jobs table are only shown if there are non-zero jobs in the table.

      Description

      MAPREDUCE-814 will provide a way to keep the job history files in HDFS. There should be a way to get the url for the completed job history fie. The completed jobs can be purged from memory more aggressively from jobtracker since the clients can retrieve the information from history file. Jobtracker can just maintain the very basic info about the completed jobs.

      1. 817_v1.patch
        60 kB
        Sharad Agarwal
      2. 817_v2.patch
        64 kB
        Sharad Agarwal
      3. 817_v3.patch
        61 kB
        Sharad Agarwal
      4. 817_ydist.patch
        49 kB
        Sharad Agarwal
      5. 817_ydist_new.patch
        38 kB
        Sharad Agarwal
      6. 817_ydist_new_1.patch
        2 kB
        Sharad Agarwal

        Issue Links

          Activity

          Sharad Agarwal created issue -
          Sharad Agarwal made changes -
          Field Original Value New Value
          Link This issue is blocked by MAPREDUCE-814 [ MAPREDUCE-814 ]
          Sharad Agarwal made changes -
          Link This issue incorporates MAPREDUCE-757 [ MAPREDUCE-757 ]
          Sharad Agarwal made changes -
          Attachment 817_v1.patch [ 12416063 ]
          Sharad Agarwal made changes -
          Attachment 817_v2.patch [ 12416180 ]
          Sharad Agarwal made changes -
          Attachment 817_v3.patch [ 12416203 ]
          Sharad Agarwal made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Sharad Agarwal made changes -
          Summary Add a JobClient API to get job history file url Add a cache for retired jobs with minimal job info and provide a way to access history file url
          Hadoop Flags [Incompatible change]
          Release Note Provides a way to configure the cache of JobStatus objects for the retired jobs. Adds an API in RunningJob to access history file url.
          Description MAPREDUCE-814 will provide a way to keep the job history files in HDFS. There should be a way to get the url for the completed job history fie. MAPREDUCE-814 will provide a way to keep the job history files in HDFS. There should be a way to get the url for the completed job history fie. The completed jobs can be purged from memory more aggressively from jobtracker since the clients can retrieve the information from history file. Jobtracker can just maintain the very basic info about the completed jobs.
          Sharad Agarwal made changes -
          Resolution Fixed [ 1 ]
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Incompatible change] [Incompatible change, Reviewed]
          Fix Version/s 0.21.0 [ 12314045 ]
          Sharad Agarwal made changes -
          Attachment 817_ydist.patch [ 12416227 ]
          Sharad Agarwal made changes -
          Attachment 817_ydist_new.patch [ 12416316 ]
          Sharad Agarwal made changes -
          Attachment 817_ydist_new_1.patch [ 12416445 ]
          Devaraj Das made changes -
          Parent MAPREDUCE-863 [ 12432997 ]
          Issue Type New Feature [ 2 ] Sub-task [ 7 ]
          Sharad Agarwal made changes -
          Release Note Provides a way to configure the cache of JobStatus objects for the retired jobs. Adds an API in RunningJob to access history file url. Provides a way to configure the cache of JobStatus objects for the retired jobs. Adds an API in RunningJob to access history file url. Adds a LRU based cache for job history files loaded in memory when accessed via JobTracker web UI.
          Sharad Agarwal made changes -
          Release Note Provides a way to configure the cache of JobStatus objects for the retired jobs. Adds an API in RunningJob to access history file url. Adds a LRU based cache for job history files loaded in memory when accessed via JobTracker web UI. Provides a way to configure the cache of JobStatus objects for the retired jobs.
          Adds an API in RunningJob to access history file url.
          Adds a LRU based cache for job history files loaded in memory when accessed via JobTracker web UI.
          Adds Retired Jobs table on the Jobtracker UI. The job move from Running to Completed/Failed table. Then job move to Retired table when it is purged from memory. The Retired table shows last 100 retired jobs. The Completed/Failed jobs table are only shown if there are non-zero jobs in the table.
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Sharad Agarwal
              Reporter:
              Sharad Agarwal
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development