[IGNITE-12204] In binary distribution, essential dependencies for ignite-spark missing

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.7.6
• Fix Version/s: 3.0
• Component/s: spark
• Ignite Flags: Release Notes Required
• Assignee: Alexey Zinoviev
• Reporter: Ilya Kasnacheev
• Created: 20/Sep/19 09:31
• Updated: 24/Jun/20 21:26

Description

It seems that we only put direct dependencies of other JARs in our binary distribution, and not transient ones.

For example, libs/optional/ignite-spark lacks the essential commons-lang3 jar, which will lead to the following error immediately:

Exception in thread "main" java.lang.NoClassDefFoundError: org.apache.commons.lang3.SystemUtils
        at org.apache.spark.util.Utils$.<init>(Utils.scala:1915)
        at org.apache.spark.util.Utils$.<clinit>(Utils.scala)
        at org.apache.spark.SparkConf.loadFromSystemProperties(SparkConf.scala:75)

It's almost impossible to fix without resorting to Maven source build.

I understand that adding Spark module to Ignite server is something not widely used, but if we ship this module at all, we should make sure that it is usable in some form.