Details
-
Sub-task
-
Status: Patch Available
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
Add property yarn.resourcemanager.decommissioning-nodes-watcher.wait-for-applications.
If true (the default), the resource manager waits for all containers, as well as all applications associated with those containers, to finish before gracefully decommissioning a node.
If false, the resource manager only waits for containers, but not applications, to finish. For map-only jobs or other jobs in which mappers do not need to serve shuffle data, this allows nodes to be decommissioned as soon as their containers are finished as opposed to when the job is done.
Add property yarn.resourcemanager.decommissioning-nodes-watcher.wait-for-app-masters.
If false, during graceful decommission, when the resource manager waits for all containers on a node to finish, it will not wait for app master containers to finish. Defaults to true. This property should only be set to false if app master failure is recoverable.