[YARN-7500] LogAggregation DeletionService should consider completedTime for long running jobs

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.7.3
• Component/s: log-aggregation
• Assignee: Prabhu Joseph
• Reporter: Prabhu Joseph
• Created: 15/Nov/17 13:38
• Updated: 16/Nov/17 08:06

Description

Currently LogAggregation deletes the application logs based on start time of the job. For long running jobs (started before yarn.log-aggregation.retain-seconds), say it is failed yesterday for some reason and we won't have the job logs today for debugging.

Better to consider the completedTime of the job as part of the deletion condition.