Details
-
Improvement
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
None
-
None
-
None
-
None
Description
The mapreduce job history server currently needs to be deployed as a trusted server in sync with the mapreduce runtime. Every new application would need a similar application history server. Having to deploy O(T*V) (where T is number of type of application, V is number of version of application) trusted servers is clearly not scalable.
Job history storage handling itself is pretty generic: move the logs and history data into a particular directory for later serving. Job history data is already stored as json (or binary avro). I propose that we create only one trusted application history server, which can have a generic UI (display json as a tree of strings) as well. Specific application/version can deploy untrusted webapps (a la AMs) to query the application history server and interpret the json for its specific UI and/or analytics.
Attachments
Attachments
Issue Links
- contains
-
YARN-1621 Add CLI to list rows of <task attempt ID, container ID, host of container, state of container>
- Resolved
- is duplicated by
-
YARN-304 RM Tracking Links for purged applications needs a long-term solution
- Resolved
- is related to
-
YARN-1701 Improve default paths of timeline store and generic history store
- Closed
-
YARN-1982 Rename the daemon name to timelineserver
- Closed
-
YARN-374 Job History Server doesn't show jobs which killed by ClientRMProtocol.forceKillApplication
- Resolved
-
YARN-1452 Document the usage of the generic application history and the timeline data service
- Closed
- relates to
-
YARN-374 Job History Server doesn't show jobs which killed by ClientRMProtocol.forceKillApplication
- Resolved
-
YARN-1530 [Umbrella] Store, manage and serve per-framework application-timeline data
- Resolved
-
YARN-2928 YARN Timeline Service v.2: alpha 1
- Resolved