Details
-
Wish
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
I understand that the list of accepted compression codecs is explicity limited to uncompressed, snappy, gzip, and lzo. (See parquet.hadoop.metadata.CompressionCodecName.java) Is there a reason for this? Or is there an easy workaround? On the surface it seems like an unnecessary restriction.
I ask because I have written a custom codec to implement encryption and I'm unable to use it with Parquet, which is a real shame because it is the main storage format I was hoping to use.
Other thoughts on how to implement encryption in Parquet with this limitation?