[YARN-543] [Umbrella] NodeManager localization related issues

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: nodemanager
• Assignee: Unassigned
• Reporter: Vinod Kumar Vavilapalli
• Created: 04/Apr/13 22:34
• Updated: 03/May/15 02:24
• Resolved: 03/May/15 02:24

Description

Seeing a bunch of localization related issues being worked on, this is the tracking ticket.

Sub-Tasks

• Jobs fail during resource localization when public distributed-cache hits unix directory limits: Closed
• Jobs fail during resource localization when private distributed-cache hits unix directory limits: Closed
• Waiting containers are not informed if private localization for a resource fails.: Resolved
• Race in localization can cause containers to fail: Closed
• LocalizedResources are leaked in memory in case resource localization fails: Closed
• Failed resource localization might introduce a race condition.: Resolved
• Race condition in Public / Private Localizer may result into resource getting downloaded again: Closed
• Shared data structures in Public Localizer and Private Localizer are not Thread safe.: Closed
• PrivateLocalizer does not support parallel resource download via ContainerLocalizer: Patch Available
• Application cache files should be localized under local-dir/usercache/userid/appcache/appid/filecache: Closed
• Consider Localizing Resources within the NM process itself in non-secure deployments: Resolved
• Public localizer crashes with "Localized unkown resource": Closed
• YARN NodeManager with java.util.concurrent.RejectedExecutionException: Closed
• NPE in public localizer: Resolved
• Killing a container that is localizing can orphan resources in the DOWNLOADING state: Closed