[IMPALA-11603] Investigate using cloudflare's zlib library

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 4.2.0
• Fix Version/s: Impala 4.3.0
• Component/s: Backend
• Epic Color: ghx-label-5
• Assignee: Joe McDonnell
• Reporter: Joe McDonnell
• Created: 21/Sep/22 17:45
• Updated: 05/May/23 18:02
• Resolved: 05/May/23 18:02

Description

Amazon recommends the use of cloudflare's zlib implementation at https://github.com/cloudflare/zlib

In a blog post, they claim pretty large performance boosts over the regular zlib implementation:

https://aws.amazon.com/blogs/opensource/improving-zlib-cloudflare-and-comparing-performance-with-other-zlib-forks/

On Arm:
  Compression performance: ~90 percent faster than zlib-madler (original zlib).
  Decompression performance: ~52 percent faster than zlib-madler.
On x86:
  Compression performance: ~113 percent faster than zlib-madler.
  Decompression performance: ~44 percent faster than zlib-madler.

The blog post is a year and a half old, so things may have changed since then, but it seems interesting. Amazon's guidebooks still recommend it.