[YARN-6565] Fix memory leak and finish app trigger in AMRMProxy

• Type: Sub-task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 2.9.0, 3.0.0-alpha4
• Hadoop Flags: Reviewed
• Assignee: Botong Huang
• Reporter: Botong Huang
• Created: 05/May/17 19:45
• Updated: 08/May/17 20:13
• Resolved: 05/May/17 23:55

Description

Two issues in AMRMProxy:

1. When application finishes, AMRMTokenSecretManager is not updated to remove related data, leading to memory leak.

2. When we kill an application, we should remove the pipeline after the AM container is killed.

FINISH_APPLICATION event is sent when the AM container is still being killed. After we remove the pipeline, we might still get heartbeats from AM, triggering exception messages.

Instead, we should wait for APPLICATION_RESOURCES_CLEANEDUP event, sent after where the AM container is killed.