[YARN-1338] Recover localized resource cache state upon nodemanager restart

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.0
• Fix Version/s: 2.5.0
• Component/s: nodemanager
• Target Version/s: 2.5.0
• Hadoop Flags: Reviewed
• Assignee: Jason Darrell Lowe Assign to me
• Reporter: Jason Darrell Lowe
• Created: 22/Oct/13 19:27
• Updated: 15/Aug/14 05:44
• Resolved: 30/May/14 16:10

Description

Today when node manager restarts we clean up all the distributed cache files from disk. This is definitely not ideal from 2 aspects.

• For work preserving restart we definitely want them as running containers are using them
• For even non work preserving restart this will be useful in the sense that we don't have to download them again if needed by future tasks.

Issue Links

• blocks: YARN-1337 Recover containers upon nodemanager restart
• requires: YARN-1987 Wrapper for leveldb DBIterator to aid in handling database exceptions