[YARN-7984] Delete registry entries from ZK on ServiceClient stop and clean up stop/destroy behavior

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 3.2.0, 3.1.1
• Component/s: yarn-native-services
• Assignee: Billie Rinaldi
• Reporter: Billie Rinaldi
• Created: 27/Feb/18 17:30
• Updated: 13/Apr/18 02:25
• Resolved: 10/Apr/18 22:08

Description

The service records written to the registry are removed by ServiceClient on a destroy call, but not on a stop call. The service AM does have some code to clean up the registry entries when component instances are stopped, but if the AM is killed before it has a chance to perform the cleanup, these entries will be left in ZooKeeper. It would be better to clean these up in the stop call, so that RegistryDNS does not provide lookups for containers that don't exist.

Additional stop/destroy behavior improvements include fixing errors / unexpected behavior related to:

• destroying a saved (not launched or started) service
• destroying a stopped service
• destroying a destroyed service
• returning proper exit codes for destroy failures
• performing other client operations on saved services (fixing NPEs)

Issue Links

• is duplicated by: YARN-8017 Validate the application ID has been persisted to the service definition prior to use
• relates to: YARN-7958 ServiceMaster should only wait for recovery of containers with id that match the current application id