Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2658

--driver-class-path can overwrite the classpath in SparkMain

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.3.0
    • Component/s: None
    • Labels:
      None

      Description

      In SparkMain we use the option --conf spark.driver.extraClassPath= to put together the jars needed for Spark. If the user specifies this option, Oozie appends to the value.
      If --driver-class-path is defined in <spark-opts> then it's value overwrites the other ones, causing the Spark action to fail.
      We should handle both ways to define classpaths the same.

        Attachments

        1. OOZIE-2658.001.patch
          5 kB
          Peter Cseh
        2. OOZIE-2658.002.patch
          5 kB
          Peter Cseh
        3. OOZIE-2658.003.patch
          5 kB
          Peter Cseh

          Activity

            People

            • Assignee:
              gezapeti Peter Cseh
              Reporter:
              gezapeti Peter Cseh
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: