[YARN-8360] Yarn service conflict between restart policy and NM configuration

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 3.2.0, 3.1.2
• Component/s: yarn
• Target Version/s: 3.2.0, 3.1.2
• Assignee: Suma Shivaprasad
• Reporter: Chandni Singh
• Created: 24/May/18 20:58
• Updated: 23/Jul/18 17:22
• Resolved: 23/Jul/18 17:15

Description

For the below spec, the service will not stop even after container failures because of the NM auto retry properties :

• "yarn.service.container-failure.retry.max": 1,
• "yarn.service.container-failure.validity-interval-ms": 5000
  The NM will continue auto-restarting containers.
  fail_after 20 fails after 20 seconds. Since the validity failure interval is 5 seconds, NM will auto restart the container.

{
  "name": "fail-demo2",
  "version": "1.0.0",
  "components" :
  [
    {
      "name": "comp1",
      "number_of_containers": 1,
      "launch_command": "fail_after 20",
      "restart_policy": "NEVER",
      "resource": {
        "cpus": 1,
        "memory": "256"
      },
      "configuration": {
        "properties": {
          "yarn.service.container-failure.retry.max": 1,
          "yarn.service.container-failure.validity-interval-ms": 5000
        }
      }
    }
  ]
}

If restart_policy is NEVER, then the service should stop after the container fails.

Since we have introduced, the service level Restart Policies, I think we should make the NM auto retry configurations part of the RetryPolicy and get rid of all yarn.service.container-failure.** properties. Otherwise it gets confusing.