[SPARK-3883] Provide SSL support for Akka and HttpServer based connections

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.3.0
• Component/s: Spark Core
• Assignee: Jacek Lewandowski
• Reporter: Jacek Lewandowski
• Created: 09/Oct/14 19:50
• Updated: 03/Feb/15 01:28
• Resolved: 03/Feb/15 01:28

Description

Spark uses at least 4 logical communication channels:
1. Control messages - Akka based
2. JARs and other files - Jetty based (HttpServer)
3. Computation results - Java NIO based
4. Web UI - Jetty based

The aim of this feature is to enable SSL for (1) and (2).

Why:
Spark configuration is sent through (1). Spark configuration may contain sensitive information like credentials for accessing external data sources or streams. Application JAR files (2) may include the application logic and therefore they may include information about the structure of the external data sources, and credentials as well.

Issue Links

• blocks: SPARK-5342 Allow long running Spark apps to run on secure YARN/HDFS
• relates to: SPARK-2750 Add Https support for Web UI
• links to: [Github] Pull Request #2739 (jacek-lewandowski)
• links to: https://github.com/apache/spark/pull/3571