[OOZIE-2705] Oozie Spark action ignores spark.executor.extraJavaOptions and spark.driver.extraJavaOptions

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 4.3.0
• Fix Version/s: 4.3.0
• Component/s: action

Environment

CentOS 6.6

• Assignee: Gézapeti
• Reporter: Prabhu Joseph
• Created: 09/Oct/16 13:57
• Updated: 02/Dec/16 21:04
• Resolved: 20/Oct/16 00:35

Description

Oozie Spark Action with spark-opts "spark.executor.extraJavaOptions and spark.driver.extraJavaOptions" are ignored.

<spark-opts>--executor-memory 1G --driver-java-options "-verbose -Xms750m -XX:+PrintGCDetails -XX:+PrintGCTimeStamps" --conf "spark.executor.extraJavaOptions=-XX:+PrintGCDetails -XX:+PrintGCTimeStamps"</spark-opts>

Suspect the OOZIE-2503 fix is overriding them. The configs are reflected after removing the fix of OOZIE-2503.

Issue Links

• is broken by: OOZIE-2503 show ChildJobURLs to spark action