[SPARK-947] Class path issue in case assembled with specific hadoop version

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.8.0
• Fix Version/s: 0.8.1, 0.9.0
• Component/s: Build
• Assignee: Unassigned
• Reporter: shekhar
• Created: 28/Oct/13 02:33
• Updated: 01/Nov/13 00:32
• Resolved: 01/Nov/13 00:32

Description

assemble with hadoop version 1.2.1 using command SPARK_HADOOP_VERSION=1.2.1 sbt/sbt clean assembly

now we have two jars $SPARK_HOME/assembly/target/scala-2.9.3/spark-assembly-0.8.0-incubating-hadoop1.2.1.jar and $SPARK_HOME/assembly/target/scala-2.9.3/spark-assembly-0.8.0-incubating-hadoop1.0.4.jar

compute class path script put both jars in classpath without adding ":" between them and results in invalid jar name.

classpath computed by script::
/data/installs/spark-0.8.0-incubating-bin-hadoop1/conf:/data/installs/spark-0.8.0-incubating-bin-hadoop1/assembly/target/scala-2.9.3/spark-assembly-0.8.0-incubating-hadoop1.0.4.jar/data/installs/spark-0.8.0-incubating-bin-hadoop1/assembly/target/scala-2.9.3/spark-assembly-0.8.0-incubating-hadoop1.2.1.jar:/data/installs/hadoop-1.2.1/conf