[SPARK-38807] Error when starting spark shell on Windows system

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.1
• Fix Version/s: 3.3.0, 3.2.2
• Component/s: Spark Core
• Assignee: Ming Li
• Reporter: Ming Li
• Created: 06/Apr/22 13:59
• Updated: 03/Jun/22 14:52
• Resolved: 02/Jun/22 12:45

Description

Using the release version of spark-3.2.1  and the default configuration, starting spark shell on Windows system fails. (spark 3.1.2 doesn't show this issue)

Here is the stack trace of the exception:

22/04/06 21:47:45 ERROR SparkContext: Error initializing SparkContext.
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        ...
        at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)
        at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
        at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.net.URISyntaxException: Illegal character in path at index 30: spark://192.168.X.X:56964/F:\classes
        at java.net.URI$Parser.fail(URI.java:2845)
        at java.net.URI$Parser.checkChars(URI.java:3018)
        at java.net.URI$Parser.parseHierarchical(URI.java:3102)
        at java.net.URI$Parser.parse(URI.java:3050)
        at java.net.URI.<init>(URI.java:588)
        at org.apache.spark.repl.ExecutorClassLoader.<init>(ExecutorClassLoader.scala:57)
        ... 70 more
22/04/06 21:47:45 ERROR Utils: Uncaught exception in thread main
java.lang.NullPointerException
            ... 

Issue Links

• is duplicated by: SPARK-38808 Windows, Spark 3.2.1: spark-shell command throwing this error: SparkContext: Error initializing SparkContext
• links to: [Github] Pull Request #36089 (1104056452)
• links to: [Github] Pull Request #36134 (1104056452)
• links to: [Github] Pull Request #36134 (1104056452)
• links to: [Github] Pull Request #36446 (1104056452)
• links to: [Github] Pull Request #36446 (1104056452)
• links to: [Github] Pull Request #36447 (1104056452)