[SPARK-27812] kubernetes client import non-daemon thread which block jvm exit.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.3, 2.4.4
• Fix Version/s: 2.4.5, 3.0.0
• Component/s: Kubernetes
• Labels: release-notes
• Assignee: Igor Calabria
• Reporter: Henry Yu
• Created: 23/May/19 06:53
• Updated: 09/Feb/20 00:25
• Resolved: 17/Oct/19 19:24

Description

I try spark-submit to k8s with cluster mode. Driver pod failed to exit with An Okhttp Websocket Non-Daemon Thread.

 

Issue Links

• is duplicated by: SPARK-28721 Failing to stop SparkSession in K8S cluster mode PySpark leaks Driver and Executors
• relates to: SPARK-26742 Bump Kubernetes Client Version to 4.1.2
• links to: GitHub Pull Request #25785
• links to: GitHub Pull Request #26093
• links to: GitHub Pull Request #26152