[SPARK-12551] Not able to load CSV package

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 1.5.2
• Fix Version/s: 1.5.2
• Component/s: SparkR

Environment

Rstudio under Windows

• External issue URL: http://stackoverflow.com/questions/33237475/sparkr-java-error
• Assignee: Unassigned
• Reporter: chintan
• Created: 29/Dec/15 09:07
• Updated: 29/Dec/15 16:15
• Resolved: 29/Dec/15 16:15

Description

When ever i am trying load CSV package spark dont work, it gives Invoke java error

Sys.setenv('SPARKR_SUBMIT_ARGS'='"--packages" "com.databricks:spark-csv_2.10:1.2.0" "sparkr-shell"')
> Sys.setenv(SPARK_MEM="1g")
> sc <- sparkR.init(master = "local")
Launching java with spark-submit command C:/spark/bin/spark-submit.cmd "--packages" "com.databricks:spark-csv_2.10:1.2.0" "sparkr-shell" C:\Users\shahch07\AppData\Local\Temp\RtmpigvXMn\backend_port98840b15c5a
> sqlContext <- sparkRSQL.init(sc)
> DF <- createDataFrame(sqlContext, faithful)
Error in invokeJava(isStatic = FALSE, objId$id, methodName, ...) :
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 (TID 0, localhost): java.lang.NullPointerException
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012)
at org.apache.hadoop.util.Shell.runCommand(Shell.java:482)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
at org.apache.hadoop.fs.FileUtil.chmod(FileUtil.java:873)
at org.apache.hadoop.fs.FileUtil.chmod(FileUtil.java:853)
at org.apache.spark.util.Utils$.fetchFile(Utils.scala:381)
at org.apache.spark.executor.Executor$$anonfun$org$apache$spark$executor$Executor$$updateDependencies$5.apply(Executor.scala:405)
at org.apache.spark.executor.Executor$$anonfun$org$apache$spark$executor$Executor$$updateDependencies$5.apply(Executor.scala:397)
at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLi