[SPARK-13447] Fix AM failure situation for dynamic allocation disabled situation

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.6.0
• Fix Version/s: 2.0.0
• Component/s: Spark Core, YARN
• Target Version/s: 2.0.0
• Assignee: Saisai Shao
• Reporter: Saisai Shao
• Created: 23/Feb/16 07:16
• Updated: 29/Mar/16 00:03
• Resolved: 29/Mar/16 00:03

Description

Because of lag of executor disconnection events, stale states in the CoarseGrainedSchedulerBacked will ruin the newly registered executor information. This situation is handled for dynamic allocation enabled situation, for dynamic allocation disabled scenario, we should also have the similar fix.

Issue Links

• relates to: SPARK-10582 using dynamic-executor-allocation, if AM failed. the new AM will be started. But the new AM does not allocate executors to dirver
• links to: [Github] Pull Request #11366 (jerryshao)