Details
-
Improvement
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
3.0.1, 3.1.1
-
None
-
None
Description
It turns out that --master k8s:// is accepted and although leads to termination displays stacktraces that don't really tell what the real cause is.
This may happen when the Kubernetes API server(s) are described by an environment variable that's not initialized in the current terminal.
$ ./bin/spark-shell --master k8s:// --verbose ... Spark config: (spark.jars,) (spark.app.name,Spark shell) (spark.submit.pyFiles,) (spark.ui.showConsoleProgress,true) (spark.submit.deployMode,client) (spark.master,k8s://https://) ... 21/01/27 14:29:44 ERROR Main: Failed to initialize Spark session. io.fabric8.kubernetes.client.KubernetesClientException: Failed to start websocket at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$1.onFailure(WatchConnectionManager.java:208) ... Caused by: java.net.UnknownHostException: api: nodename nor servname provided, or not known at java.base/java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:929) at java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1519) at java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:848) at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1509) at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1368) at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1302) at okhttp3.Dns$1.lookup(Dns.java:40) at okhttp3.internal.connection.RouteSelector.resetNextInetSocketAddress(RouteSelector.java:185)