[YARN-431] [Umbrella] Complete/Stabilize YARN application log-handling

• Type: Task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Vinod Kumar Vavilapalli
• Created: 27/Feb/13 19:53
• Updated: 28/Aug/15 02:47

Issue Links

• relates to: MAPREDUCE-3143 Complete aggregation of user-logs spit out by containers onto DFS
• relates to: YARN-4029 Update LogAggregationStatus to store on finish

Sub-Tasks

• Allow per job log aggregation configuration: Open
• NodeManager should serve logs directly if log-aggregation is not enabled: Open
• Log aggregation should not assume an AppMaster will have containerId 1: Open
• Documentation for Log Aggregation and log retrieval.: Open
• NM should limit number of applications who's logs are being aggregated: Open
• nodemanager log aggregation has scaling issues with namenode: Open
• Page navigation support for container logs page and the logs web-service on NMs: Open
• Yarn aggregated logs are difficult for external tools to understand: Open
• IOException in AppLogAggregatorImpl does not give stacktrace and leaves aggregated TFile in a bad state.: Open
• Ability to limit or avoid aggregating logs beyond a certain size: Open
• Log aggregation finish time should get logged for trouble shooting.: Open