[SPARK-42895] ValueError when invoking any session operations on a stopped Spark session

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.5.0
• Fix Version/s: 3.4.0
• Component/s: Connect
• Assignee: Allison Wang
• Reporter: Allison Wang
• Created: 22/Mar/23 08:58
• Updated: 29/Mar/23 08:02
• Resolved: 29/Mar/23 08:02

Description

If a remote Spark session is stopped, trying to invoke any session operations will result in a ValueError. For example:

 

spark.stop()
spark.sql("select 1")

ValueError: Cannot invoke RPC: Channel closed!
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  ...
    return e.code() == grpc.StatusCode.UNAVAILABLE
AttributeError: 'ValueError' object has no attribute 'code'

 

Issue Links

• links to: [Github] Pull Request #40536 (allisonwang-db)