[HADOOP-16550] Spark config name error on the Launching Applications Using Docker Containers page

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.9.0, 2.8.2, 2.8.3, 3.0.0, 3.1.0, 2.9.1, 3.0.1, 2.8.4, 3.0.2, 3.1.1, 2.9.2, 3.0.3, 2.8.5, 3.1.2
• Component/s: documentation
• Assignee: Unassigned
• Reporter: Attila Zsolt Piros
• Created: 05/Sep/19 10:32
• Updated: 06/Sep/19 08:46
• Resolved: 06/Sep/19 08:46

Description

On the "Launching Applications Using Docker Containers" page at the "Example: Spark" section the Spark config for configuring the environment variables for the application master the config prefix are wrong:

• spark.yarn.AppMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE
• spark.yarn.AppMasterEnv.YARN_CONTAINER_RUNTIME_TYPE

The correct ones:

• spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE
• spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_TYPE

See https://spark.apache.org/docs/2.4.0/running-on-yarn.html:

spark.yarn.appMasterEnv.[EnvironmentVariableName]

Issue Links

• links to: GitHub Pull Request #9