[SPARK-14845] spark.files in properties file is not distributed to driver in yarn-cluster mode

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: Spark Core, Spark Submit, YARN
• Assignee: Jeff Zhang
• Reporter: Jeff Zhang
• Created: 22/Apr/16 10:02
• Updated: 17/May/20 18:16
• Resolved: 02/May/16 18:04

Description

I use the following command to run SparkPi example. And in this property file, define spark.files. It turns out that in yarn-cluster mode this README.md is only distirbuted to executor but not in driver.
content of properties file

spark.files=/Users/jzhang/github/spark/README.md

launch command

bin/spark-submit --master yarn-cluster --properties-file a.properties --class org.apache.spark.examples.SparkPi examples/target/original-spark-examples_2.11-2.0.0-SNAPSHOT.jar

Issue Links

• is duplicated by: SPARK-10436 spark-submit overwrites spark.files defaults with the job script filename
• links to: [Github] Pull Request #12656 (zjffdu)