[SPARK-6491] Spark will put the current working dir to the CLASSPATH

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0
• Fix Version/s: 1.3.1
• Component/s: Spark Submit
• Assignee: Liangliang Gu
• Reporter: Liangliang Gu
• Created: 24/Mar/15 06:41
• Updated: 26/Mar/15 13:30
• Resolved: 26/Mar/15 13:30

Description

When running "bin/computer-classpath.sh", the output will be:

:/spark/conf:/spark/assembly/target/scala-2.10/spark-assembly-1.3.0-hadoop2.5.0-cdh5.2.0.jar:/spark/lib_managed/jars/datanucleus-rdbms-3.2.9.jar:/spark/lib_managed/jars/datanucleus-api-jdo-3.2.6.jar:/spark/lib_managed/jars/datanucleus-core-3.2.10.jar

Java will add the current working dir to the CLASSPATH, if the first ":" exists, which is not expected by spark users.

For example, if I call spark-shell in the folder /root. And there exists a "core-site.xml" under /root/. Spark will use this file as HADOOP CONF file, even if I have already set HADOOP_CONF_DIR=/etc/hadoop/conf.

Issue Links

• relates to: SPARK-4831 Current directory always on classpath with spark-submit
• links to: [Github] Pull Request #5156 (marsishandsome)