Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Incomplete
-
2.4.3
-
None
-
None
Description
Spark Master could potentially persist data via ZooKeeperPersistenceEngine even if it is not the leader. See the execution order below.
1. master-1 became the leader.
2. master-1 received message and wanted to addApplication(or addWorker)
3. master-1 stuck because of a full gc
4. master-1 lost leadership on zk. master-2 became the leader.
5. master-1 received RevokedLeadership message but the message was pending.
6. master-1 finished persisting data.