[IMPALA-8450] Add support for zstd in parquet

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 3.3.0
• Component/s: Backend
• Labels: parquet
• Target Version: Impala 3.3.0
• Epic Color: ghx-label-5
• Assignee: Abhishek Rawat
• Reporter: Tim Armstrong
• Created: 24/Apr/19 18:32
• Updated: 05/Jun/19 17:18
• Resolved: 05/Jun/19 17:18

Description

PARQUET-970 added these codecs to the format. We have LZ4 in the toolchain already and I just added zstd: https://gerrit.cloudera.org/#/c/13079/

These codec probably offer a better trade-off of density and speed than snappy or gzip.

https://github.com/apache/arrow/pull/807/files might be a useful crib sheet for how to add a compressor.

LZ4 support will be added using: https://issues.apache.org/jira/browse/IMPALA-8617