[SPARK-2750] Add Https support for Web UI

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: Web UI
• Labels: https ssl webui
• Assignee: Fei Wang
• Reporter: Tao Wang (Inactive)
• Created: 30/Jul/14 13:04
• Updated: 15/Feb/19 23:45
• Resolved: 19/Jan/16 22:51

Description

Now I try to add https support for web ui using Jetty ssl integration.Below is the plan:
1.Web UI include Master UI, Worker UI, HistoryServer UI and Spark Ui. User can switch between https and http by configure "spark.http.policy" in JVM property for each process, while choose http by default.
2.Web port of Master and worker would be decided in order of launch arguments, JVM property, System Env and default port.
3.Below is some other configuration items:

spark.ssl.server.keystore.location The file or URL of the SSL Key store
spark.ssl.server.keystore.password  The password for the key store
spark.ssl.server.keystore.keypassword The password (if any) for the specific key within the key store
spark.ssl.server.keystore.type The type of the key store (default "JKS")
spark.client.https.need-auth True if SSL needs client authentication
spark.ssl.server.truststore.location The file name or URL of the trust store location
spark.ssl.server.truststore.password The password for the trust store
spark.ssl.server.truststore.type The type of the trust store (default "JKS")

Any feedback is welcome!

Issue Links

• is related to: SPARK-3883 Provide SSL support for Akka and HttpServer based connections
• relates to: SPARK-24621 WebUI - application 'name' urls point to http instead of https (even when ssl enabled)
• links to: [Github] Pull Request #1714 (WangTaoTheTonic)
• links to: [Github] Pull Request #1980 (scwf)
• links to: [Github] Pull Request #5664 (WangTaoTheTonic)
• links to: [Github] Pull Request #10238 (vanzin)