Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
Impala 4.0.0
-
ghx-label-4
Description
Impala currently uses zstd version 1.4.0. It looks like there are some performance improvements in more recent versions:
v1.4.5 perf: Improved decompression speed: x64 : +10% (clang) / +5% (gcc); ARM : from +15% to +50%, depending on SoC, by @terrelln perf: Automatically downsizes ZSTD_DCtx when too large for too long (#2069, by @bimbashreshta) perf: Improved fast compression speed on aarch64 (#2040, ~+3%, by @caoyzh) perf: Small level 1 compression speed gains (depending on compiler) v1.4.4 perf: Improved decompression speed, by > 10%, by @terrelln perf: Better compression speed when re-using a context, by @felixhandte perf: Fix compression ratio when compressing large files with small dictionary, by @senhuang42 perf: zstd reference encoder can generate RLE blocks, by @bimbashrestha perf: minor generic speed optimization, by @davidbolvansky v1.4.1 perf: Improve decode speed by ~7% @mgrice (#1668) perf: Slightly improved compression ratio of level 3 and 4 (ZSTD_dfast) by @cyan4973 (#1681) perf: Slightly faster compression speed when re-using a context by @cyan4973 (#1658) perf: Improve compression ratio for small windowLog by @cyan4973 (#1624) perf: Faster compression speed in high compression mode for repetitive data by @terrelln (#1635)
https://github.com/facebook/zstd/blob/dev/CHANGELOG