Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2705

Oozie Spark action ignores spark.executor.extraJavaOptions and spark.driver.extraJavaOptions

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 4.3.0
    • Fix Version/s: 4.3.0
    • Component/s: action
    • Labels:
      None
    • Environment:

      CentOS 6.6

      Description

      Oozie Spark Action with spark-opts "spark.executor.extraJavaOptions and spark.driver.extraJavaOptions" are ignored.

      <spark-opts>--executor-memory 1G --driver-java-options "-verbose -Xms750m -XX:+PrintGCDetails -XX:+PrintGCTimeStamps" --conf "spark.executor.extraJavaOptions=-XX:+PrintGCDetails -XX:+PrintGCTimeStamps"</spark-opts>

      Suspect the OOZIE-2503 fix is overriding them. The configs are reflected after removing the fix of OOZIE-2503.

        Attachments

        1. OOZIE-2705.003.patch
          4 kB
          Gézapeti
        2. OOZIE-2705.002.patch
          4 kB
          Gézapeti
        3. OOZIE-2705.001.patch
          2 kB
          Gézapeti

          Issue Links

            Activity

              People

              • Assignee:
                gezapeti Gézapeti
                Reporter:
                prabhujoseph Prabhu Joseph
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: