[YARN-10123] Error message around yarn app -stop/start can be improved to highlight that an implementation at framework level is needed for the stop/start functionality to work

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.2.1
• Fix Version/s: 3.4.0, 3.2.3, 3.3.2
• Component/s: client, documentation
• Assignee: Siddharth Ahuja
• Reporter: Siddharth Ahuja
• Created: 10/Feb/20 07:05
• Updated: 06/Oct/22 08:44
• Resolved: 22/May/21 12:40

Description

A "stop" on a YARN application fails with the below error:

# yarn app -stop application_1581294743321_0002 -appTypes SPARK
20/02/10 06:24:27 INFO client.RMProxy: Connecting to ResourceManager at c3224-node2.squadron.support.hortonworks.com/172.25.34.128:8050
20/02/10 06:24:27 INFO client.AHSProxy: Connecting to Application History server at c3224-node2.squadron.support.hortonworks.com/172.25.34.128:10200
Exception in thread "main" java.lang.IllegalArgumentException: App admin client class name not specified for type SPARK
        at org.apache.hadoop.yarn.client.api.AppAdminClient.createAppAdminClient(AppAdminClient.java:76)
        at org.apache.hadoop.yarn.client.cli.ApplicationCLI.run(ApplicationCLI.java:579)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
        at org.apache.hadoop.yarn.client.cli.ApplicationCLI.main(ApplicationCLI.java:123)

From https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/api/AppAdminClient.java#L76, it seems that this is because user does not have the setting:

yarn.application.admin.client.class.SPARK

set up in their client configuration.

However, even if this setting is present, we still need to have an implementation available for the application type. From my internal discussions - Jobs don't have a notion of stop / resume functionality at YARN level. If some apps like Spark need it, it has to be implemented at those framework's level.

Therefore, the above error message is a bit misleading in that, even if "yarn.application.admin.client.class.SPARK" is supplied (or for that matter - yarn.application.admin.client.class.MAPREDUCE), if there is no implementation actually available underneath to handle the stop/start functionality then, we will fail again, albeit with a different error here: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/api/AppAdminClient.java#L85.

As such, maybe this error message can be potentially improved to say something like:

Exception in thread "main" java.lang.IllegalArgumentException: App admin client class name not specified for type SPARK. Please ensure the App admin client class actually exists within SPARK to handle this functionality.

or something similar.

Further, documentation around "-stop" and "-start" options will need to be improved here -> https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YarnCommands.html#application_or_app as it does not mention anything about having an implementation at the framework level for the YARN stop/start command to succeed.