[OOZIE-2658] --driver-class-path can overwrite the classpath in SparkMain

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 4.3.0
• Assignee: Gézapeti
• Reporter: Gézapeti
• Created: 30/Aug/16 18:51
• Updated: 02/Dec/16 21:04
• Resolved: 11/Oct/16 04:07

Description

In SparkMain we use the option --conf spark.driver.extraClassPath= to put together the jars needed for Spark. If the user specifies this option, Oozie appends to the value.
If --driver-class-path is defined in <spark-opts> then it's value overwrites the other ones, causing the Spark action to fail.
We should handle both ways to define classpaths the same.