[MAPREDUCE-4680] Job history cleaner should only check timestamps of files in old enough directories

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0-alpha
• Fix Version/s: 2.3.0
• Component/s: jobhistoryserver
• Hadoop Flags: Reviewed
• Assignee: Robert Kanter
• Reporter: Sandy Ryza
• Created: 24/Sep/12 22:17
• Updated: 24/Feb/14 20:57
• Resolved: 28/Oct/13 23:56

Description

Job history files are stored in yyyy/mm/dd folders. Currently, the job history cleaner checks the modification date of each file in every one of these folders to see whether it's past the maximum age. The load on HDFS could be reduced by only checking the ages of files in directories that are old enough, as determined by their name.