Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-3050

Parse Error if the spark job has the spark.executor.extraClassPath confguration.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 4.1.0
    • Fix Version/s: None
    • Component/s: client, core, workflow
    • Labels:
      None

      Description

      1. create a spark job workflow in Oozie
      2. add --conf spark.executor.extraClassPath=/XXX/XXX in job confuration
      3. submit the job
      Results:
      the Oozie workflow failed and the error log is as follows. it has an addtion error "--conf".
      --conf
      --conf
      spark.executor.extraClassPath=aaa:$PWD/*
      --conf
      spark.driver.extraClassPath=$PWD/*

      Error: Spark config without '=': --conf
      Run with --help for usage help or --verbose for debug output
      Intercepting System.exit(1)
      Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], exit code [1]

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                Tang Yan Tang Yan
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: