[MAPREDUCE-3973] [Umbrella JIRA] JobHistoryServer performance improvements in YARN+MR

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.23.0
• Component/s: jobhistoryserver, mrv2
• Assignee: Unassigned
• Reporter: Vinod Kumar Vavilapalli
• Created: 05/Mar/12 22:12
• Updated: 11/Jun/15 16:23

Description

Few parallel efforts are happening w.r.t improving/fixing issues with JobHistoryServer in MR over YARN. This is the umbrella ticket so we have the complete picture.

Sub-Tasks

• JobHistory web services are slower then the UI and can easly overload the JH: Resolved
• Locking and exception issues in JobHistory Server.: Closed
• Job History web services need to have limits on the number of items they can return.: Open
• Add separate cache for CompletedJobs with tasks loaded and without tasks loaded in JobHIstoryServer: Open
• Add the equivalent of JobStatus to end of JobHistory file: Resolved
• Add server-side pagination to JobHistoryServer tasks page: Resolved
• Add avro binary support for jhist files: Resolved
• Speed up Task processing loop in HsTasksBlock#render(): Resolved