Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
Add master high availability support for Yarn. The idea is to let Yarn restart a failed application master in a new container. For that, we set the number of application retries to something greater than 1.
From version 2.4.0 onwards, it is possible to reuse already started containers for the TaskManagers, thus, avoiding unnecessary restart delays.
From version 2.6.0 onwards, it is possible to specify an interval in which the number of application attempts have to be exceeded in order to fail the job. This will prevent long running jobs from eventually depleting all available application attempts.