[YARN-6302] Fail the node if Linux Container Executor is not configured properly

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 2.9.0, 3.0.0-alpha4
• Hadoop Flags: Reviewed
• Assignee: Miklos Szegedi
• Reporter: Miklos Szegedi
• Created: 07/Mar/17 21:08
• Updated: 05/Mar/18 00:16
• Resolved: 19/Apr/17 19:25

Description

We have a cluster that has one node with misconfigured Linux Container Executor. Every time an AM or regular container is launched on the cluster, it will fail. The node will still have resources available, so it keeps failing apps until the administrator notices the issue and decommissions the node. AM Blacklisting only helps, if the application is already running.

As a possible improvement, when the LCE is used on the cluster and a NM gets certain errors back from the LCE, like error 24 configuration not found, we should not try to allocate anything on the node anymore or shut down the node entirely. That kind of problem normally does not fix itself and it means that nothing can really run on that node.

Application application_1488920587909_0010 failed 2 times due to AM Container for appattempt_1488920587909_0010_000002 exited with exitCode: -1000
Failing this attempt.Diagnostics: Application application_1488920587909_0010 initialization failed (exitCode=24) with output:
For more detailed output, check the application tracking page: http://node-1.domain.com:8088/cluster/app/application_1488920587909_0010 Then click on links to logs of each attempt.
. Failing the application.

Issue Links

• is related to: YARN-6475 Fix some long function checkstyle issues
• links to: GitHub Pull Request #200