[HUDI-238] Make separate release for hudi spark/scala based packages for scala 2.12

• Type: Improvement
• Status: Closed
• Priority: Blocker
• Resolution: Unresolved
• Fix Version/s: 0.5.1
• Component/s: Release & Administrative, Usability
• Labels: pull-request-available
• Assignee: Tadas Sugintas
• Reporter: Balaji Varadarajan
• Created: 06/Sep/19 04:17
• Updated: 23/Feb/20 07:09

Description

https://github.com/apache/incubator-hudi/issues/881#issuecomment-528700749

Suspects: 

Hudi utilities package 

bringing in spark-streaming-kafka-0.8* 

[INFO] Scanning for projects...
[INFO] 
[INFO] -------------------< org.apache.hudi:hudi-utilities >-------------------
[INFO] Building hudi-utilities 0.5.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ hudi-utilities ---
[INFO] org.apache.hudi:hudi-utilities:jar:0.5.0-SNAPSHOT
[INFO] ...
[INFO] +- org.apache.hudi:hudi-client:jar:0.5.0-SNAPSHOT:compile
       ...
[INFO] 
[INFO] +- org.apache.hudi:hudi-spark:jar:0.5.0-SNAPSHOT:compile
[INFO] |  \- org.scala-lang:scala-library:jar:2.11.8:compile
[INFO] +- log4j:log4j:jar:1.2.17:compile
       ...
[INFO] +- org.apache.spark:spark-core_2.11:jar:2.1.0:provided
[INFO] |  +- org.apache.avro:avro-mapred:jar:hadoop2:1.7.7:provided
[INFO] |  |  +- org.apache.avro:avro-ipc:jar:1.7.7:provided
[INFO] |  |  \- org.apache.avro:avro-ipc:jar:tests:1.7.7:provided
[INFO] |  +- com.twitter:chill_2.11:jar:0.8.0:provided
[INFO] |  +- com.twitter:chill-java:jar:0.8.0:provided
[INFO] |  +- org.apache.xbean:xbean-asm5-shaded:jar:4.4:provided
[INFO] |  +- org.apache.spark:spark-launcher_2.11:jar:2.1.0:provided
[INFO] |  +- org.apache.spark:spark-network-common_2.11:jar:2.1.0:provided
[INFO] |  +- org.apache.spark:spark-network-shuffle_2.11:jar:2.1.0:provided
[INFO] |  +- org.apache.spark:spark-unsafe_2.11:jar:2.1.0:provided
[INFO] |  +- net.java.dev.jets3t:jets3t:jar:0.7.1:provided
[INFO] |  +- org.apache.curator:curator-recipes:jar:2.4.0:provided
[INFO] |  +- org.apache.commons:commons-lang3:jar:3.5:provided
[INFO] |  +- org.apache.commons:commons-math3:jar:3.4.1:provided
[INFO] |  +- com.google.code.findbugs:jsr305:jar:1.3.9:provided
[INFO] |  +- org.slf4j:slf4j-api:jar:1.7.16:compile
[INFO] |  +- org.slf4j:jul-to-slf4j:jar:1.7.16:provided
[INFO] |  +- org.slf4j:jcl-over-slf4j:jar:1.7.16:provided
[INFO] |  +- org.slf4j:slf4j-log4j12:jar:1.7.16:compile
[INFO] |  +- com.ning:compress-lzf:jar:1.0.3:provided
[INFO] |  +- org.xerial.snappy:snappy-java:jar:1.1.2.6:compile
[INFO] |  +- net.jpountz.lz4:lz4:jar:1.3.0:compile
[INFO] |  +- org.roaringbitmap:RoaringBitmap:jar:0.5.11:provided
[INFO] |  +- commons-net:commons-net:jar:2.2:provided
       ....
[INFO] +- org.apache.spark:spark-sql_2.11:jar:2.1.0:provided
[INFO] |  +- com.univocity:univocity-parsers:jar:2.2.1:provided
[INFO] |  +- org.apache.spark:spark-sketch_2.11:jar:2.1.0:provided
[INFO] |  \- org.apache.spark:spark-catalyst_2.11:jar:2.1.0:provided
[INFO] |     +- org.codehaus.janino:janino:jar:3.0.0:provided
[INFO] |     +- org.codehaus.janino:commons-compiler:jar:3.0.0:provided
[INFO] |     \- org.antlr:antlr4-runtime:jar:4.5.3:provided
[INFO] +- com.databricks:spark-avro_2.11:jar:4.0.0:provided
[INFO] +- org.apache.spark:spark-streaming_2.11:jar:2.1.0:compile
[INFO] +- org.apache.spark:spark-streaming-kafka-0-8_2.11:jar:2.1.0:compile
[INFO] |  \- org.apache.kafka:kafka_2.11:jar:0.8.2.1:compile
[INFO] |     +- org.scala-lang.modules:scala-xml_2.11:jar:1.0.2:compile
[INFO] |     +- org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.2:compile
[INFO] |     \- org.apache.kafka:kafka-clients:jar:0.8.2.1:compile
[INFO] +- io.dropwizard.metrics:metrics-core:jar:4.0.2:compile
[INFO] +- org.antlr:stringtemplate:jar:4.0.2:compile
[INFO] |  \- org.antlr:antlr-runtime:jar:3.3:compile
[INFO] +- com.beust:jcommander:jar:1.72:compile
[INFO] +- com.twitter:bijection-avro_2.11:jar:0.9.2:compile
[INFO] |  \- com.twitter:bijection-core_2.11:jar:0.9.2:compile
[INFO] +- io.confluent:kafka-avro-serializer:jar:3.0.0:compile
[INFO] +- io.confluent:common-config:jar:3.0.0:compile
[INFO] +- io.confluent:common-utils:jar:3.0.0:compile
[INFO] |  \- com.101tec:zkclient:jar:0.5:compile
[INFO] +- io.confluent:kafka-schema-registry-client:jar:3.0.0:compile
[INFO] \- org.mockito:mockito-all:jar:1.10.19:test
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  2.211 s
[INFO] Finished at: 2019-09-12T09:39:32-07:00
[INFO] ------------------------------------------------------------------------
 

Issue Links

• links to: GitHub Pull Request #1109
• links to: GitHub Pull Request #1226