[SPARK-38074] RuntimeError: Java gateway process exited before sending its port number

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Invalid
• Affects Version/s: 3.2.1
• Component/s: PySpark, Spark Shell, Spark Submit
• Assignee: Unassigned
• Reporter: Malla
• Created: 30/Jan/22 17:12
• Updated: 12/Dec/22 18:10
• Resolved: 03/Feb/22 05:01

Description

I am getting 

RuntimeError: Java gateway process exited before sending its port number when running python tests in  Docker.  

 

I am using spark_home as /usr/lib/python3.9/site-packages/pyspark

 

sc = SparkContext() File "/usr/lib/python3.9/site-packages/pyspark/context.py", line 144, in _init_ SparkContext._ensure_initialized(self, gateway=gateway, conf=conf) File

"/usr/lib/python3.9/site-packages/pyspark/context.py", line 339, in _ensure_initialized

SparkContext._gateway = gateway or launch_gateway(conf)

File "/usr/lib/python3.9/site-packages/pyspark/java_gateway.py", line 136, in launch_gateway raise RuntimeError("Java gateway process exited before sending its port number")

RuntimeError: Java gateway process exited before sending its port number

 

I can provide additional details. Any Help is appreciated