[MAPREDUCE-3143] Complete aggregation of user-logs spit out by containers onto DFS

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.23.0
• Fix Version/s: 0.23.0
• Component/s: mrv2, nodemanager
• Assignee: Unassigned
• Reporter: Vinod Kumar Vavilapalli
• Created: 05/Oct/11 12:30
• Updated: 01/Sep/13 23:28
• Resolved: 02/Nov/11 10:05

Description

Already implemented the feature for handling user-logs spit out by containers in NodeManager. But the feature is currently disabled due to user-interface issues.

This is the umbrella ticket for tracking the pending bugs w.r.t putting container-logs on DFS.

Issue Links

• is related to: YARN-431 [Umbrella] Complete/Stabilize YARN application log-handling
• relates to: YARN-85 Allow per job log aggregation configuration
• relates to: YARN-432 Documentation for Log Aggregation and log retrieval.
• relates to: MAPREDUCE-3738 NM can hang during shutdown if AppLogAggregatorImpl thread dies unexpectedly
• relates to: MAPREDUCE-3977 LogAggregationService leaks log aggregator objects
• relates to: MAPREDUCE-3398 Log Aggregation broken in Secure Mode
• relates to: MAPREDUCE-3298 Improvements to the aggregated log format

Sub-Tasks

• Augment JobHistory to include information needed for serving aggregated logs.: Closed
• Fix NM UI to serve logs from DFS once application finishes: Closed
• JobHistory should link to task logs: Closed
• Add a MR specific command line to dump logs for a given TaskAttemptID: Closed
• Container logs aren't getting cleaned up when LogAggregation is disabled: Closed