[SPARK-19139] AES-based authentication mechanism for Spark

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.0
• Fix Version/s: 2.2.0
• Component/s: Spark Core
• Assignee: Marcelo Masiero Vanzin
• Reporter: Marcelo Masiero Vanzin
• Created: 09/Jan/17 21:57
• Updated: 24/Jan/17 18:44
• Resolved: 24/Jan/17 18:44

Description

In SPARK-13331, support for AES encryption was added to the Spark network library. But the authentication of different Spark processes is still performed using SASL's DIGEST-MD5 mechanism. That means the authentication part is the weakest link; since the AES keys are currently encrypted using 3des (strongest cipher supported by SASL), Spark can't really claim to provide the full benefits of using AES for encryption.

We should add a new auth protocol that doesn't need these disclaimers.

Issue Links

• links to: [Github] Pull Request #16521 (vanzin)