-
Type:
Bug
-
Status: Open
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: 2.3.2, 2.4.0
-
Fix Version/s: None
-
Component/s: Deploy
-
Labels:None
In spark root pom, there is a test-jar execution for all sub-projects including spark-streaming. This will attach an artifact: org.apache.spark:spark-streaming_2.11:test-jar:tests:2.3.2
Also, in streaming pom, there is a shade test configuration, it will attach an artifact: org.apache.spark:spark-streaming_2.11:jar:tests:2.3.2
But two artifacts actually point to the same file: spark-streaming_2.11-2.3.2-tests.jar
So, when deploy spark to nexus using mvn deploy, maven will upload the test.jar twice since it belongs to 2 artifacts. Then deploy fails due to nexus don't allow overrides existing file for non-SNAPSHOT release.
What's more, after checking the spark-streaming, shaded test jar is exactly same to original test jar. It seems we can just delete the related shade config.
<build>
<outputDirectory>target/scala-${scala.binary.version}/classes</outputDirectory>
<testOutputDirectory>target/scala-${scala.binary.version}/test-classes</testOutputDirectory>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<configuration>
<shadeTestJar>true</shadeTestJar>
</configuration>
</plugin>
</plugins>
</build>