[YARN-5063] Fail to launch AM continuously on a lost NM

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: resourcemanager
• Assignee: Jun Gong
• Reporter: Jun Gong
• Created: 09/May/16 12:55
• Updated: 16/May/16 02:48

Description

If a NM node shuts down, RM will not mark it as LOST until liveness monitor finds it timeout. However before that, RM might continuously allocate AM on that NM.

We found this case in our cluster: RM continuously allocated a same AM on a lost NM before RM found it lost, and AMLauncher always failed because it could not connect to the lost NM. To solve the problem, we could add the NM to AM blacklist if RM failed to launch it.

Issue Links

• is related to: YARN-4837 User facing aspects of 'AM blacklisting' feature need fixing