[SPARK-44417] Spark Driver and executors are not transferring from standby master to Alive Master.

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 3.3.2
• Component/s: Deploy

Environment

RedHat linux 

Zookeeper HA Cluster

• Assignee: Unassigned
• Reporter: Anand Changediya
• Created: 14/Jul/23 05:24
• Updated: 14/Jul/23 05:26

Description

Issue with the Spark driver and executors transferring from Standby Master to Alive Master.

Spark is deployed with Zookeeper as cluster manager. Ideally if standby node(server) goes down it should transfer applications running on standby node to alive node.
Application transformations are killed in the Standby Master.
  # Just consider we have two Application drivers were running on the Standby Master server.

1. If we forcefully shutdown, This should trigger the failover scenario and application should move to Alive node. But, The Standby Master node both the application drivers were not transferring to Active master instead those were killing.

Check attachment for reference.
 
Steps to reproduce

• Deploy Spark HA cluster with Zookeeper
• Deploy multiple application until we have drivers and executor assigned on standby node.
• Now reboot the server
• Check if all the applications were moving to Active Master.