[YARN-171] NodeManager should serve logs directly if log-aggregation is not enabled

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.23.3
• Component/s: nodemanager
• Assignee: Siddharth Seth Assign to me
• Reporter: Vinod Kumar Vavilapalli
• Created: 18/Oct/12 23:59
• Updated: 17/Jun/13 23:12

Description

NodeManagers never serve logs for completed applications. If log-aggregation is not enabled, in the interim, due to bugs like YARN-162, this is a serious problem for users as logs are necessarily not available.

We should let nodes serve logs directly if YarnConfiguration.LOG_AGGREGATION_ENABLED is set. This should be okay as NonAggregatingLogHandler can retain logs upto YarnConfiguration.NM_LOG_RETAIN_SECONDS.

Issue Links

• blocks: MAPREDUCE-4734 The history server should link back to NM logs if aggregation is incomplete / disabled
• is related to: YARN-162 nodemanager log aggregation has scaling issues with namenode