[SPARK-17622] Cannot run create or load DF on Windows- Spark 2.0.0

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Not A Bug
• Affects Version/s: 2.0.0
• Component/s: SparkR
• Labels: windows

Environment

windows 10
R 3.3.1
RStudio 1.0.20

• Assignee: Unassigned
• Reporter: renzhi he
• Created: 21/Sep/16 11:45
• Updated: 22/Sep/16 15:43
• Resolved: 22/Sep/16 15:43

Description

Under spark2.0.0- on Windows- when try to load or create data with the similar codes below, I also get error message and cannot execute the functions.

sc <- sparkR.session(master="local",sparkConfig = list(spark.driver.memory = "2g"))

df <- as.DataFrame(faithful)

Here is the error message:
#Error in invokeJava(isStatic = TRUE, className, methodName, ...) :
#java.lang.reflect.InvocationTargetException
#at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
#at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
#at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
#at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
#at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:258)
#at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:359)
#at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:263)
#at org.apache.spark.sql.hive.HiveSharedState.metadataHive$lzycompute(HiveSharedState.scala:39)
#at org.apache.spark.sql.hive.HiveSharedState.metadataHive(HiveSharedState.scala:38)
#at org.apache.spark.sql.hive.HiveSharedState.externalCatalog$lzycompute(HiveSharedState.scala:46)
#at org.apache.spark.sql.hive.HiveSharedSt

However, under spark1.6.1 or spark1.6.2, run the same functional functions, there will be no problem.

sc1 <- sparkR.init(master = "local", sparkEnvir = list(spark.driver.memory="2g"))

sqlContext <- sparkRSQL.init(sc1)

df <- as.DataFrame(sqlContext,faithful