[YARN-4094] Add Configration to support encryption of Distributed Cache Data

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.6.0, 2.7.0
• Component/s: documentation, yarn
• Assignee: Unassigned
• Reporter: Vijay Singh
• Created: 30/Aug/15 09:01
• Updated: 04/Jan/16 06:41

Description

Currently Ditributed cache does not allow mechanism to encrypt the data that gets copied over during processing. One attack vector is to process small files that contain sensitive data to use this mechanism to access contents of small files.
This requests aims to counter that by providing for configuration at service level that lets yarn encrypt all the data that gets to cache on each node. Yarn components should encrypt while copying the data on to disk and decrypt during the processing. Lets start by leveraging the symmetric key mechanism used by HDFS transparent mechanism similar to DEK (Data Encryption key) that could be generated as part of the process.
Next step could be to setup Encryption zone key similar to transperent encryption mechanism.
Please suggest if there is a better way.