Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-20338

Spaces in spark.eventLog.dir are not correctly handled

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.0
    • Fix Version/s: 2.3.0
    • Component/s: Spark Core
    • Labels:
      None

      Description

      set spark.eventLog.dir=/home/mr/event log and submit an app ,we got error as follows:
      017-04-14 17:28:40,378 INFO org.apache.spark.SparkContext: Successfully stopped SparkContext
      Exception in thread "main" ExitCodeException exitCode=1: chmod: cannot access `/home/mr/event%20log/app-20170414172839-0000.inprogress': No such file or directory

      at org.apache.hadoop.util.Shell.runCommand(Shell.java:561)
      at org.apache.hadoop.util.Shell.run(Shell.java:478)
      at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:738)
      at org.apache.hadoop.util.Shell.execCommand(Shell.java:831)
      at org.apache.hadoop.util.Shell.execCommand(Shell.java:814)
      at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:712)
      at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:506)
      at org.apache.spark.scheduler.EventLoggingListener.start(EventLoggingListener.scala:125)
      at org.apache.spark.SparkContext.<init>(SparkContext.scala:516)
      at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2258)
      at org.apache.spark.sql.SparkSession$Builder$$anonfun$9.apply(SparkSession.scala:879)
      at org.apache.spark.sql.SparkSession$Builder$$anonfun$9.apply(SparkSession.scala:871)
      at scala.Option.getOrElse(Option.scala:121)
      at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:871)
      at org.apache.spark.sql.hive.thriftserver.SparkSQLEnv$.init(SparkSQLEnv.scala:58)
      at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.<init>(SparkSQLCLIDriver.scala:288)
      at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:137)
      at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:736)
      at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:185)
      at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:210)
      at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:124)
      at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

        Attachments

          Activity

            People

            • Assignee:
              zuo.tingbing9 zuotingbing
              Reporter:
              zuo.tingbing9 zuotingbing
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: