Description
A simple Spark application that uses both Spark and Ignite fails to compile in sbt under 2.11 due to conflicting dependencies. For a simple sbt definition with:
libraryDependencies += "org.apache.spark" %% "spark-core" % "2.1.0" libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.1.0" libraryDependencies += "org.apache.ignite" % "ignite-spark" % "2.1.0"
it will fail to compile with:
[error] Modules were resolved with conflicting cross-version suffixes in: [error] org.scalatest:scalatest _2.11, _2.10 [error] com.twitter:chill _2.11, _2.10 [error] org.apache.spark:spark-unsafe _2.11, _2.10 [error] org.apache.spark:spark-tags _2.11, _2.10 [trace] Stack trace suppressed: run 'last *:update' for the full output.
It looks like a single culprit is the entry in ignite-spark's pom.xml for
spark-unsafe_2.10.
When removed, compiled and published, everything works great.
I don't see why such an entry exists in spark module when there is a separate spark-2.10 module as well.
Happy to submit a PR if anyone is willing to give a thumbs up.