[SPARK-32023] Generate spark streaming test jar with maven plugin maven-jar-plugin

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Cannot Reproduce
• Affects Version/s: 3.0.0
• Component/s: Structured Streaming
• Assignee: Unassigned
• Reporter: Wan Kun
• Created: 18/Jun/20 12:18
• Updated: 27/Jun/20 02:03
• Resolved: 27/Jun/20 02:03

Description

When I compile streaming module alone, the test jar generated incorrect.
So we need to import maven plugin `maven-jar-plugin`

mvn clean package -DskipTests -Phive -Phive-thriftserver -Pyarn -DskipTests -pl streaming

[kun.wan@nta-offline-gw1 streaming]$ ll target/ -l
total 2668
drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:46 analysis
drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:46 antrun
drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:46 maven-archiver
drwxr-xr-x 3 kun.wan staff 4096 Jun 18 19:45 maven-shared-archive-resources
-rw-r--r-- 1 kun.wan staff 1134932 Jun 18 19:46 original-spark-streaming_2.12-3.0.0.jar
drwxr-xr-x 4 kun.wan staff 4096 Jun 18 19:46 scala-2.12
-rw-r--r-- 1 kun.wan staff 1137992 Jun 18 19:46 spark-streaming_2.12-3.0.0.jar
-rw-r--r-- 1 kun.wan staff 252535 Jun 18 19:46 spark-streaming_2.12-3.0.0-sources.jar
-rw-r--r-- 1 kun.wan staff 22 Jun 18 19:46 spark-streaming_2.12-3.0.0-tests.jar
-rw-r--r-- 1 kun.wan staff 170004 Jun 18 19:46 spark-streaming_2.12-3.0.0-test-sources.jar
drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:46 tmp
# After change maven plugin
[kun.wan@nta-offline-gw1 streaming]$ ll target/ -l
total 3500
drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:52 analysis
drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:51 antrun
drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:52 maven-archiver
drwxr-xr-x 3 kun.wan staff 4096 Jun 18 19:51 maven-shared-archive-resources
-rw-r--r-- 1 kun.wan staff 1134959 Jun 18 19:52 original-spark-streaming_2.12-3.0.0.jar
drwxr-xr-x 4 kun.wan staff 4096 Jun 18 19:51 scala-2.12
-rw-r--r-- 1 kun.wan staff 1138019 Jun 18 19:52 spark-streaming_2.12-3.0.0.jar
-rw-r--r-- 1 kun.wan staff 252558 Jun 18 19:52 spark-streaming_2.12-3.0.0-sources.jar
-rw-r--r-- 1 kun.wan staff 853239 Jun 18 19:52 spark-streaming_2.12-3.0.0-tests.jar
-rw-r--r-- 1 kun.wan staff 170027 Jun 18 19:52 spark-streaming_2.12-3.0.0-test-sources.jar
drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:51 tmp

 

 

Issue Links

• links to: [Github] Pull Request #28857 (wankunde)
• links to: [Github] Pull Request #28857 (wankunde)