Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2658

--driver-class-path can overwrite the classpath in SparkMain

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • None
    • 4.3.0
    • None
    • None

    Description

      In SparkMain we use the option --conf spark.driver.extraClassPath= to put together the jars needed for Spark. If the user specifies this option, Oozie appends to the value.
      If --driver-class-path is defined in <spark-opts> then it's value overwrites the other ones, causing the Spark action to fail.
      We should handle both ways to define classpaths the same.

      Attachments

        1. OOZIE-2658.001.patch
          5 kB
          Gézapeti
        2. OOZIE-2658.002.patch
          5 kB
          Gézapeti
        3. OOZIE-2658.003.patch
          5 kB
          Gézapeti

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            gezapeti Gézapeti
            gezapeti Gézapeti
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment