Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Incomplete
-
2.4.0
-
None
Description
This works in PySpark,
from pyspark.sql import SparkSession spark = SparkSession.builder \ .master('k8s://https://kubernetes.default') \
But in Scala,
val spark = SparkSession.builder()
.master("k8s://https://kubernetes.default")
I get the following exception,
Caused by: org.apache.spark.SparkException: Could not parse Master URL: 'k8s://https://kubernetes.default'
Attachments
Attachments
Issue Links
- is duplicated by
-
SPARK-28244 Problem to recognize Kubernetes cluster address from Scala
- Resolved