Details
-
New Feature
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
2.4.0
-
None
Description
As we all know that spark on Yarn uses DB to record RegisteredExecutors information which can be reloaded and used again when the ExternalShuffleService is restarted .
The RegisteredExecutors information can't be recorded both in the mode of spark's standalone and spark on k8s , which will cause the RegisteredExecutors information to be lost ,when the ExternalShuffleService is restarted.
To solve the problem above, a method is proposed and is committed .