[PARQUET-1376] Data obfuscation layer for encryption

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Gidon Gershinsky
• Reporter: Gidon Gershinsky
• Created: 10/Aug/18 20:49
• Updated: 04/Aug/20 08:19

Description

Data obfuscation in sensitive columns - for users without access to column encryption keys.

1. Implement on top of basic Parquet encryption 
2. Built-in support for multiple masking mechanisms, with different trade-off between data utility, leakage, and size/throughput overhead
3. Provide interface for plug-in custom masking mechanism
4. Enable storing multiple masked versions of the same column in a file
5. Provide readers with explicit list of column’s masked versions in a file
6. Enable readers to select a masked version of a column
7. Stretch: Implement tools for analysis of file data privacy properties and information leakage
8. Stretch: Leverage privacy analysis tools for tuning file data anonymity
9. Optional: Support aggregated obfuscation

Issue Links

• depends upon: PARQUET-1178 Parquet modular encryption
• links to: discussion googledoc