[SPARK-38808] Windows, Spark 3.2.1: spark-shell command throwing this error: SparkContext: Error initializing SparkContext

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.2.1
• Component/s: Spark Shell
• Assignee: Unassigned
• Reporter: Taras
• Created: 06/Apr/22 14:29
• Updated: 16/Apr/22 20:02
• Resolved: 16/Apr/22 20:02

Description

Can't start spark-shell on Windows for Spark 3.2.1. Downgrading Spark version to 3.1.3 fixed the problem. I need Pandas API on Spark, which is available on >3.2, so it's not a solution for me.

The bug and the workaround are described in the link below as well

https://stackoverflow.com/questions/69923603/spark-shell-command-throwing-this-error-sparkcontext-error-initializing-sparkc

Issue Links

• duplicates: SPARK-38807 Error when starting spark shell on Windows system