[YARN-539] LocalizedResources are leaked in memory in case resource localization fails

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.1.0-beta
• Hadoop Flags: Reviewed
• Assignee: Omkar Vinit Joshi
• Reporter: Omkar Vinit Joshi
• Created: 03/Apr/13 20:47
• Updated: 27/Aug/13 22:15
• Resolved: 11/Apr/13 02:09

Description

If resource localization fails then resource remains in memory and is
1) Either cleaned up when next time cache cleanup runs and there is space crunch. (If sufficient space in cache is available then it will remain in memory).
2) reused if LocalizationRequest comes again for the same resource.

I think when resource localization fails then that event should be sent to LocalResourceTracker which will then remove it from its cache.

Issue Links

• blocks: YARN-537 Waiting containers are not informed if private localization for a resource fails.
• blocks: YARN-544 Failed resource localization might introduce a race condition.
• duplicates: YARN-537 Waiting containers are not informed if private localization for a resource fails.
• duplicates: YARN-544 Failed resource localization might introduce a race condition.