gzip vs zlib; benchmarking considerations

[SolidComp]

Hi all – The Large Text Compression Benchmark led me down a winding road on tracking down the canonical gzip and zlib applications. The LTCB uses an ancient version of gzip for Windows, which is almost certainly slower than current zlib for generating gzip files, and probably slower than GNU gzip. The gzip for Windows in the LTCB was released in 2006 or possibly earlier. Using that old build is therefore misleading with respect to how gzip or zlib actually performs compared to other codecs.

If you want to include gzip or zlib in your benchmarks, these are the relevant websites:

GNU gzip:

http://savannah.gnu.org/projects/gzip/
https://www.gnu.org/software/gzip/
http://www.gzip.org/

zlib:

https://www.zlib.net/
https://github.com/madler/zlib

As you know, zlib is generally used to generate gzip files, typically on web servers. zlib can also generate a zlib file format, but no one seems interested in it (is it another DEFLATE wrapper?), since browsers consume gzip.

zlib is a library used for streaming applications, like the above mentioned web servers (nginx, Apache, IIS, H2O).

GNU gzip is an end-user application included in most Linux and BSD distributions for compressing files as needed. (Is it called by other applications?) They have three different websites, as listed above.

It might be worth including both in benchmarks (the latest versions).

[jibz]

As you know, zlib is generally used to generate gzip files, typically on web servers. zlib can also generate a zlib file format, but no one seems interested in it (is it another DEFLATE wrapper?), since browsers consume gzip.

I believe PNG files store their deflate compressed data in zlib format.

[JamesB]

You may also want to consider libdeflate or zlibng:

https://github.com/ebiggers/libdeflate
https://github.com/zlib-ng/zlib-ng

There are CloudFlare and Intel optimised versions of zlib too.

Edit: for block based formats libdeflate is super. I added it to htslib for the "BAM" sequence alignment format. It beat both cloudflare and intel versions and trounced the original zlib, but bizarrely all linux distributions seem to still ship with the slow code base. Libdeflate however isn't a drop-in replacement (unlike the others). https://github.com/samtools/htslib/pull/581

[Jyrki Alakuijala]

Streaming and non-streaming implementations/formats should not be mixed in benchmarking, as often properly streaming are about 2x slower but lead to better overall system performance (like web pages loading faster, intermediate results shown earlier, subsequent queries issued earlier hiding overall latency).

For gzip's main use (web) a non-streaming implementation would likely be highly harmful for system performance.

[SolidComp]

You may also want to consider libdeflate or zlibng:

https://github.com/ebiggers/libdeflate
https://github.com/zlib-ng/zlib-ng

There are CloudFlare and Intel optimised versions of zlib too.

Edit: for block based formats libdeflate is super. I added it to htslib for the "BAM" sequence alignment format. It beat both cloudflare and intel versions and trounced the original zlib, but bizarrely all linux distributions seem to still ship with the slow code base. Libdeflate however isn't a drop-in replacement (unlike the others). https://github.com/samtools/htslib/pull/581

Yes, libdeflate is very good. The issue with those projects is that none are drop-in replacements for zlib, so they have very limited use. libdeflate doesn't do streaming. zlib-ng was declared by its contributors as not production ready last time I checked. Cloudflare doesn't document their forks and provides no instructions, so they usually don't work. The Intel fork didn't work either when the Centminmod project tried to use it (Cloudflare wouldn't build either).

[SolidComp]

Yes, libdeflate is very good. The issue with those projects is that none are drop-in replacements for zlib, so they have very limited use. libdeflate doesn't do streaming. zlib-ng was declared by its contributors as not production ready last time I checked. Cloudflare doesn't document their forks and provides no instructions, so they usually don't work. The Intel fork didn't work either when the Centminmod project tried to use it (Cloudflare wouldn't build either).

Well, it looks like he ultimately got Cloudflare's fork working: https://community.centminmod.com/thr...nx-zlib.14084/

[SolidComp]

Streaming and non-streaming implementations/formats should not be mixed in benchmarking, as often properly streaming are about 2x slower but lead to better overall system performance (like web pages loading faster, intermediate results shown earlier, subsequent queries issued earlier hiding overall latency).

For gzip's main use (web) a non-streaming implementation would likely be highly harmful for system performance.

That's a good point. I think a good use for libdeflate and zopfli is precompressing static files, like for Static Site Generators like Jekyll, and SVG images, etc.