[MAPREDUCE-5025] Key Distribution and Management for supporting crypto codec in Map Reduce

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: security
• Assignee: Haifeng Chen
• Reporter: Haifeng Chen
• Created: 25/Feb/13 03:37
• Updated: 10/Mar/15 04:30

Description

This task defines the work to enable Map Reduce to utilize the Crypto Codec framework to support encryption and decryption of data during MapReduce Job.

According to the some real use case and discussions from the community, for encryption and decryption files in Map Reduce, we have the following requirements:
1. Different stages (input, output, intermediate output) should have the flexibility to choose whether encrypt or not, as well as which crypto codec to use.
2. Different stages may have different scheme of providing the keys.
3. Different Files (for example, different input files) may have or use different keys.
4. Support a flexible way of retrieving keys for encryption or decryption.

So this task defines and provides the framework for supporting these requirements as well as the implementations for common use and key retrieving scenarios.

The design document of this part is included in the Hadoop Crypto Design attached in HADOOP-9331.

Issue Links

• depends upon: HADOOP-9331 Hadoop crypto codec framework and crypto codec implementations
• is required by: PIG-3289 Encryption aware load and store functions
• is required by: HIVE-5207 Support data encryption for Hive tables