[SPARK-29790] Add notes about port being required for Kubernetes API URL when set as master

• Type: Documentation
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.4.3, 2.4.4
• Fix Version/s: 2.4.5, 3.0.0
• Component/s: Kubernetes, Spark Core
• Assignee: Emil Sandstø
• Reporter: Emil Sandstø
• Created: 07/Nov/19 13:50
• Updated: 17/May/20 18:25
• Resolved: 08/Nov/19 17:37

Description

Apparently, when configuring the master endpoint, the Kubernetes API url needs to include the port, even if it's 443. This should be noted in the documentation of "Running Spark on Kubernetes guide".

Reported in the wild https://medium.com/@kidane.weldemariam_75349/thanks-james-on-issuing-spark-submit-i-run-into-this-error-cc507d4f8f0d

I had the same issue myself.

We might want to create an issue for fixing the implementation, if it's considered a bug.

Issue Links

• links to: GitHub Pull Request #26426