[YARN-7282] Shared Cache Phase 2

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Chris Trezzo
• Created: 02/Oct/17 20:34
• Updated: 13/Jun/19 15:12

Description

Phase 2 will address more features that need to be built as part of the shared cache project. See YARN-1492 for the first release of the shared cache.

Issue Links

• is related to: YARN-7001 If shared cache upload is terminated in the middle, the temp file will never be deleted
• relates to: YARN-9616 Shared Cache Manager Failed To Upload Unpacked Resources
• relates to: YARN-1492 truly shared cache for jars (jobjar/libjar)

Sub-Tasks

• Race condintion in shared cache CleanerTask during deletion of resource: Open
• shared cache service should authorize calls properly: Patch Available
• Add support for directories in the Shared Cache: Open
• support more flexible policy for uploading in shared cache: Open
• Improve YARN shared cache support for LinuxContainerExecutor: Open
• Optionally run shared cache manager as part of the resource manager: Open