Description
Currently when running in Standalone mode, Spark UI's link to workers and application drivers are pointing to internal/protected network endpoints. So to access workers/application UI user's machine has to connect to VPN or need to have access to internal network directly.
Therefore the proposal is to make Spark master UI reverse proxy this information back to the user. So only Spark master UI needs to be opened up to internet.
The minimal changes can be done by adding another route e.g. http://spark-master.com/target/<endpoint>/ so when request goes to target, ProxyServlet kicks in and takes the <endpoint> and forwards the request to it and send response back to user.
More information about discussions for this features can be found on this mailing list thread http://apache-spark-developers-list.1001551.n3.nabble.com/spark-on-kubernetes-tc17599.html