[SPARK-50164] Spark Kubernetes support requires vulnerable okhttp dependency

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.4.1, 3.5.3
• Component/s: Kubernetes

Environment

Spark 3.4.1/Spark 3.5.3, on Ubuntu

• Assignee: Unassigned
• Reporter: Kevin Bowman
• Created: 29/Oct/24 21:57
• Updated: 29/Oct/24 21:57

Description

We have been trying to resolve the following CVE flagged in our Spark install: https://www.cve.org/CVERecord?id=CVE-2023-0833

The vulnerability is on okhttp-3.12.12.jar.  It's pulled in by the fabric8 kubernetes client: kubernetes-client-6.7.2.jar.

The fabric8 kubernetes client does support using other HTTP clients. However, Spark is explicitly initializing it with a bespoke okhttp client instance. We are using 3.4.1, but the problem is still there in the master branch.

1. Creating an okhttp dispatcher with a custom apache thread pool
2. Building an okhttp client factory with that dispatcher
3. Initializing fabric8 kubernetes client with the custom okhttp client factory

Furthermore, we are unable to upgrade the okhttp client library itself.
The earliest version of okhttp that does not have the CVE, and is also not dependent on a version of the Kotlin standard library that doesn't have CVEs, appears to be okhttp:4.10.0 (with kotlin-stdlib:1.6.20). However, the latest release of the fabric8 kubernetes client (}kubernetes-client:6.13.4) is still dependent on okhttp 3.12.12. It cannot run with okhttp 4.10.

In summary: Because it's hard-coded into Spark we are unable to replace okhttp, and because fabric8 kubernetes client stopped support for newer versions of okhttp we are unable to upgrade it to a non-vulnerable version.