[YARN-544] Failed resource localization might introduce a race condition.

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Assignee: Omkar Vinit Joshi
• Reporter: Omkar Vinit Joshi
• Created: 04/Apr/13 22:56
• Updated: 11/Apr/13 22:36
• Resolved: 11/Apr/13 22:35

Description

When resource localization fails [Public localizer / LocalizerRunner(Private)] it sends ContainerResourceFailedEvent to the containers which then sends ResourceReleaseEvent to the failed resource. In the end when LocalizedResource's ref count drops to 0 its state is changed from DOWNLOADING to INIT.
Now if a Resource gets ResourceRequestEvent in between ContainerResourceFailedEvent and last ResourceReleaseEvent then for that resource ref count will not drop to 0 and the container which sent the ResourceRequestEvent will keep waiting.

Issue Links

• is blocked by: YARN-539 LocalizedResources are leaked in memory in case resource localization fails
• is blocked by: YARN-547 Race condition in Public / Private Localizer may result into resource getting downloaded again
• is duplicated by: YARN-539 LocalizedResources are leaked in memory in case resource localization fails
• is duplicated by: YARN-547 Race condition in Public / Private Localizer may result into resource getting downloaded again