[SPARK-28504] Web UI does not always honor spark.ui.proxyBase and X-Forwarded-Context

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.4.3
• Component/s: Web UI
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Kyle Winkelman
• Created: 24/Jul/19 16:50
• Updated: 25/May/21 01:52
• Resolved: 25/May/21 01:38

Description

When running the Web UI behind a proxy, links in the html correctly honor spark.ui.proxyBase and X-Forwarded-Context in [UIUtils#uiRoot|https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/ui/UIUtils.scala].

Two items that I have found that do not honor spark.ui.proxyBase and X-Forwarded-Context:

1. History server links from the main page produce a request like http://localhost:8888/proxy/18080/history/spark-application-1563978242545/1/jobs/ which will then redirect to http://localhost:8888/history/spark-application-1563975642858/jobs/.
2. Static redirects like when entering the Web UI with a request like http://localhost:8888/proxy/4040 which will then redirect to http://localhost:8888/jobs.

I believe that these correspond to:

1. HistoryServer#loaderServlet
2. JettyUtils#createRedirectHandler

I do not know if this is a complete list but I think these instances can be fixed by using UIUtils.uiRoot.

// HistoryServer
val requestURI = UIUtils.uiRoot(req) + Option(req.getQueryString).map("?" + _).getOrElse("")

// JettyUtils
val newUrl = new URL(new URL(UIUtils.uiRoot(request)), prefixedDestPath).toString