Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-29790

Add notes about port being required for Kubernetes API URL when set as master

    XMLWordPrintableJSON

    Details

    • Type: Documentation
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.4.3, 2.4.4
    • Fix Version/s: 2.4.5, 3.0.0
    • Component/s: Kubernetes, Spark Core
    • Labels:
      None

      Description

      Apparently, when configuring the master endpoint, the Kubernetes API url needs to include the port, even if it's 443. This should be noted in the documentation of "Running Spark on Kubernetes guide".

      Reported in the wild https://medium.com/@kidane.weldemariam_75349/thanks-james-on-issuing-spark-submit-i-run-into-this-error-cc507d4f8f0d

      I had the same issue myself.

      We might want to create an issue for fixing the implementation, if it's considered a bug.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                tapped Emil Sandstø
                Reporter:
                tapped Emil Sandstø
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: