We(chenchun) are testing RM work preserving restart and found the following logs when we ran a simple MapReduce task "PI". NM continuously reported completed containers whose Application had already finished while AM had finished.
In the patch for
YARN-1372, ApplicationImpl on NM should guarantee to clean up already completed applications. But it will only remove appId from 'app.context.getApplications()' when ApplicaitonImpl received evnet 'ApplicationEventType.APPLICATION_LOG_HANDLING_FINISHED' , however NM might receive this event for a long time or could not receive.
- For NonAggregatingLogHandler, it wait for YarnConfiguration.NM_LOG_RETAIN_SECONDS which is 3 * 60 * 60 sec by default, then it will be scheduled to delete Application logs and send the event.
- For LogAggregationService, it might fail(e.g. if user does not have HDFS write permission), and it will not send the event.