For the below spec, the service will not stop even after container failures because of the NM auto retry properties :
- "yarn.service.container-failure.retry.max": 1,
- "yarn.service.container-failure.validity-interval-ms": 5000
The NM will continue auto-restarting containers.
fail_after 20 fails after 20 seconds. Since the validity failure interval is 5 seconds, NM will auto restart the container.
If restart_policy is NEVER, then the service should stop after the container fails.
Since we have introduced, the service level Restart Policies, I think we should make the NM auto retry configurations part of the RetryPolicy and get rid of all yarn.service.container-failure.** properties. Otherwise it gets confusing.