When dealing with sensitive data, it is required to keep the data encrypted wherever it is stored. Common use case is to pull encrypted data out of a datasource and store in HDFS for analysis. The keys are stored in an external keystore.
The feature adds a customizable framework to integrate different types of keystores, support for Java KeyStore, read keys from keystores, and transport keys from JobClient to Tasks.
The feature adds PGP encryption as a codec and additional utilities to perform encryption related steps.
The design document is attached. It explains the requirement, design and use cases.
Kindly review and comment. Collaboration is very much welcome.
I have a tested patch for this for 1.1 and will upload it soon as an initial work for further refinement.
Update: The patches are uploaded to subtasks.