[IMPALA-13387] Add ability to compile against zlib-ng

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: Impala 4.5.0
• Component/s: Infrastructure
• Epic Color: ghx-label-10
• Assignee: Unassigned
• Reporter: Joe McDonnell
• Created: 16/Sep/24 18:32
• Updated: 16/Sep/24 18:32

Description

Regular zlib doesn't take advantage of CPU features like AVX, etc. Impala already supports building against Cloudflare zlib, but Cloudflare zlib is irregularly maintained. For example, Cloudflare zlib doesn't not implement crc32_z, which has been in regular Zlib for several years. This means that it can't be used to compile the Mold linker.

Zlib-ng is more regularly maintained, and Fedora has talked about shipping it as the default Zlib. Zlib-ng has incorporated the performance fixes that were in Cloudflare zlib, so it should be just as fast (if not faster).

We should support building against it, and it should probably replace Cloudflare zlib as the default that we ship.