[YARN-4216] Container logs not shown for newly assigned containers after NM recovery

• Type: Bug
• Status: Patch Available
• Priority: Critical
• Resolution: Unresolved
• Component/s: log-aggregation, nodemanager
• Labels: oct16-medium
• Assignee: Bibin Chundatt
• Reporter: Bibin Chundatt
• Created: 01/Oct/15 10:09
• Updated: 27/Oct/16 18:46

Description

Steps to reproduce

1. Start 2 nodemanagers with NM recovery enabled
2. Submit pi job with 20 maps
3. Once 5 maps gets completed in NM 1 stop NM (yarn daemon stop nodemanager)
   (Logs of all completed container gets aggregated to HDFS)
4. Now start the NM1 again and wait for job completion
   The newly assigned container logs on NM1 are not shown

hdfs log dir state

1. When logs are aggregated to HDFS during stop its with NAME (localhost_38153)
2. On log aggregation after starting NM the newly assigned container logs gets uploaded with name (localhost_38153.tmp)

History server the logs are now shown for new task attempts