[SPARK-16742] Kerberos support for Spark on Mesos

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.3.0
• Component/s: Mesos
• Assignee: Arthur Rand
• Reporter: Michael Gummelt
• Created: 26/Jul/16 21:09
• Updated: 25/Aug/17 22:16
• Resolved: 17/Aug/17 22:50

Description

We at Mesosphere have written Kerberos support for Spark on Mesos. We'll be contributing it to Apache Spark soon.

Mesosphere design doc: https://docs.google.com/document/d/1xyzICg7SIaugCEcB4w1vBWp24UDkyJ1Pyt2jtnREFqc/edit#heading=h.tdnq7wilqrj6
Mesosphere code: https://github.com/mesosphere/spark/commit/73ba2ab8d97510d5475ef9a48c673ce34f7173fa

Issue Links

• is blocked by: SPARK-20434 Move Hadoop delegation token code from yarn to core
• is related to: SPARK-20328 HadoopRDDs create a MapReduce JobConf, but are not MapReduce jobs
• is related to: SPARK-5158 Allow for keytab-based HDFS security in Standalone mode
• is related to: SPARK-20812 Add Mesos Secrets support to the spark dispatcher
• relates to: SPARK-20060 Support Standalone visiting secured HDFS
• relates to: SPARK-19143 API in Spark for distributing new delegation tokens (Improve delegation token handling in secure clusters)
• links to: [Github] Pull Request #16788 (arinconstrio)
• links to: [Github] Pull Request #17665 (mgummelt)
• links to: [Github] Pull Request #18519 (mgummelt)