[YARN-2261] YARN should have a way to run post-application cleanup

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: resourcemanager
• Assignee: Vinod Kumar Vavilapalli
• Reporter: Vinod Kumar Vavilapalli
• Created: 08/Jul/14 06:29
• Updated: 29/Aug/17 21:42

Description

See MAPREDUCE-5956 for context. Specific options are at https://issues.apache.org/jira/browse/MAPREDUCE-5956?focusedCommentId=14054562&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14054562.

Issue Links

• duplicates: YARN-5759 Capability to register for a notification/callback on the expiry of timeouts
• is depended upon by: YARN-7125 Revisit deleteService call once YARN is able to do post clean up for an app
• is depended upon by: YARN-5855 DELETE call sometimes returns success when app is not deleted
• is depended upon by: SLIDER-1143 Clean Intermediate data (generated while apps running),when destry the app
• is related to: MAPREDUCE-5956 MapReduce AM should not use maxAttempts to determine if this is the last retry
• is related to: MAPREDUCE-4099 ApplicationMaster may fail to remove staging directory
• is related to: YARN-5658 YARN should have a hook to delete a path from HDFS when an application ends

Sub-Tasks

• Add user specified information for clean-up container in ApplicationSubmissionContext: Open
• Find a way to reserve AMContainer resource to launch clean-up container in CapacityScheduler: Open
• Add a CLEANUP state in RMApp to track the status of clean-up container.: Open