[HADOOP-9997] Improve TFile API to be able to pass the context for encryption codecs

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.0-alpha1
• Component/s: io
• Labels: Rhino
• Assignee: Unassigned
• Reporter: Haifeng Chen
• Created: 26/Sep/13 07:32
• Updated: 12/May/16 18:27

Description

HADOOP-9996 extends TFile to support any compression codecs available in Hadoop, such as Snappy. This also provides the opportunity to utilize the crypto codec for encryption, a new feature added by HADOOP-9331. The crypto codec is based on the compression codec and it is a natural extension of its usage here. While in API level encryption usually needs a key context while compression usually doesn’t.

This work needs to extend the TFile Reader and Writer to passing and handling a context of the codec. For crypto codecs, this may be the key information for using to encrypt or decrypt the data. To keep the backward compatibility of the reader and writer, new methods or constructor with codec context can be added instead of modifying the existing methods.

Issue Links

• is required by: PIG-3289 Encryption aware load and store functions
• requires: HADOOP-9331 Hadoop crypto codec framework and crypto codec implementations
• requires: HADOOP-9996 Improve TFile format to support any compression codecs