[MAPREDUCE-1302] TrackerDistributedCacheManager can delete file asynchronously

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.20.2, 0.21.0, 0.22.0
• Fix Version/s: 0.21.0
• Component/s: tasktracker
• Hadoop Flags: Reviewed
• Assignee: Zheng Shao
• Reporter: Zheng Shao
• Created: 16/Dec/09 07:37
• Updated: 24/Aug/10 21:19
• Resolved: 12/Jan/10 19:52

Description

With the help of AsyncDiskService from MAPREDUCE-1213, we should be able to delete files from distributed cache asynchronously.

That will help make task initialization faster, because task initialization calls the code that localizes files into the cache and may delete some other files.
The deletion can slow down the task initialization speed.

Issue Links

• duplicates: MAPREDUCE-1141 Localization of a task's distributed-cache files gets blocked by deletion of old unrelated files when mapreduce.tasktracker.cache.local.size is hit.
• is blocked by: MAPREDUCE-1213 TaskTrackers restart is very slow because it deletes distributed cache directory synchronously