[SPARK-3330] Successive test runs with different profiles fail SparkSubmitSuite

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 1.0.2
• Component/s: Build
• Assignee: Unassigned
• Reporter: Sean R. Owen
• Created: 31/Aug/14 17:14
• Updated: 02/Sep/14 22:05
• Resolved: 02/Sep/14 22:05

Description

Maven-based Jenkins builds have been failing for a while:
https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-Master-Maven-with-YARN/480/HADOOP_PROFILE=hadoop-2.4,label=centos/console

One common cause is that on the second and subsequent runs of "mvn clean test", at least two assembly JARs will exist in assembly/target. Because assembly is not a submodule of parent, "mvn clean" is not invoked for assembly. The presence of two assembly jars causes spark-submit to fail.

Issue Links

• links to: [Github] Pull Request #2221 (srowen)