[OOZIE-2677] Multiple jline version jars in spark sharelib

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: core
• Assignee: Unassigned
• Reporter: Abhishek Bafna
• Created: 19/Sep/16 13:59
• Updated: 19/Sep/16 14:01

Description

Oozie Spark sharelib is packing multiple version of the jilne jar. I have not produce/re-produced any specific issue because of the different versions of the jline, but in general, It is not a good thing. [The Oozie tar was prepared with hadoop-2 profile.]

ab:share abafna$ pwd
/Users/abafna/workspace/apache/oozie/distro/target/oozie-4.3.0-SNAPSHOT/share
ab:share abafna$ find . -name jline*
./lib/hive/jline-0.9.94.jar
./lib/hive2/jline-0.9.94.jar
./lib/pig/jline-0.9.94.jar
./lib/spark/jline-0.9.94.jar
./lib/spark/jline-2.10.5.jar

Here is the dependency tree from spark module for jline. Full output is attached in the bug.

[INFO] +- org.apache.spark:spark-repl_2.10:jar:1.6.1:compile
[INFO] |  +- org.scala-lang:scala-compiler:jar:2.10.5:compile
[INFO] |  +- org.scala-lang:scala-reflect:jar:2.10.5:compile
[INFO] |  \- org.scala-lang:jline:jar:2.10.5:compile
[INFO] |     \- org.fusesource.jansi:jansi:jar:1.4:compile


[INFO] +- org.apache.spark:spark-hive_2.10:jar:1.6.1:compile
[INFO] |  +- com.twitter:parquet-hadoop-bundle:jar:1.6.0:compile
[INFO] |  +- org.spark-project.hive:hive-cli:jar:1.2.1.spark:compile
[INFO] |  |  +- commons-cli:commons-cli:jar:1.2:compile
[INFO] |  |  \- jline:jline:jar:0.9.94:compile
[INFO] |  +- org.apache.avro:avro:jar:1.7.7:compile