Investigate SIMD-accelerated zlib implementations

[kleisauke]

It might be worth switching to a performance-oriented fork of the popular zlib library within the prebuild libvips binaries. Several companies have made zlib significantly faster by using modern CPU instructions (SSE, ARM NEON).

The zlib-ng project seems the most promising, it has merged the changes of both Cloudflare and Intel forks and cleans up the code in general. It's API / ABI compatible with zlib (when configured with --zlib-compat or -DZLIB_COMPAT=ON), so it can be used as a drop-in replacement (just like libjpeg-turbo).

There is no "stable" release for zlib-ng yet, so it might be too risky to migrate now. However, it's already been continuously fuzzed by OSS-Fuzz^[1] and the Chromium / AOSP developers are also interested in switching to this fork^[2][3].

I've already done some experiments with zlib-ng within the MXE-based Windows builds^[4], and see a nice performance improvement by converting a 4,000 x 5,000 JPEG image to PNG with different compression values^[5].

^[1] https://github.com/google/oss-fuzz/tree/master/projects/zlib-ng
^[2] https://crbug.com/687631
^[3] https://groups.google.com/a/chromium.org/d/msg/blink-dev/T3rPfd6G6ps/l8pAk7gnBgAJ
^[4] https://github.com/libvips/build-win64-mxe/tree/zlib-ng-experiment
^[5] https://gist.github.com/kleisauke/879e3075a675c6f945dff3772d2ef0a3

[lovell]

Thanks, I think we'd want to wait for a stable/supported release before making the switch.

I recall some of the Cloudflare changes were GPL-licenced but it doesn't look like these made it across.

[lovell]

There's a release candidate of zlib-ng at https://github.com/zlib-ng/zlib-ng/releases/tag/v2.0.0-RC1

[lovell]

I've created a WIP branch that switches to the latest zlib-ng release candidate (RC2).

https://github.com/lovell/sharp-libvips/actions/runs/607358257

The sharp tests pass locally on Linux x64 using the resultant binary too.