[YARN-4233] YARN Timeline Service plugin: ATS v1.5

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: timelineserver
• Assignee: Xuan Gong
• Reporter: Xuan Gong
• Created: 07/Oct/15 17:14
• Updated: 04/Nov/16 01:06

Description

Copy the description from YARN-3942: Timeline store to read events from HDFS
This adds a new timeline store plugin that is intended as a stop-gap measure to mitigate some of the issues we've seen with ATS v1 while waiting for ATS v2. The intent of this plugin is to provide a workable solution for running the Tez UI against the timeline server on a large-scale clusters running many thousands of jobs per day.

Issue Links

• depends upon: YARN-4772 Overloaded leveljb can crash the ATS "pthread lock: Invalid argument"
• is related to: YARN-4705 ATS 1.5 parse pipeline to consider handling open() events recoverably
• is related to: YARN-4774 ATS can block in shutdown under heavy load
• is related to: HADOOP-12906 AuthenticatedURL should convert a 404/Not Found into an FileNotFoundException.
• relates to: YARN-5835 NoClassDefFoundError when reading ATS v1.5

Sub-Tasks

• Allow YARN system metrics publisher to use ATS v1.5 APIs: Open
• Document ATS v1.5: Open
• ATS 1.5 parse pipeline to consider handling open() events recoverably: Open