libdeflate - a new optimized DEFLATE implementation

[JamesB]

For what it's worth, we added this to the Samtools/Htslib packages for manipulating DNA sequencing files: https://github.com/samtools/htslib/pull/581

The speed gains were very much worth it. We also fuzz tested with American Fuzzy Lop and found no issues. A big thumbs up from me.

The only thing I can see lacking, which sadly I didn't get around to fixing, is a proper "make install" target.

[JamesB]

Test of libdeflate-1.0-windows-x86_64-bin

1.787.785.811 mkv
1.786.957.948 mkv.7z.22.zstd - 137 sec
1.785.465.763 mkv.winrar.best.zip - 27 sec
1.785.419.731 mkv.7z.deflate.mx.zip - 11.5 min
1.785.400.203 mkv.1.gz - 37 sec
1.785.221.871 mkv.7z.deflate64.mx.zip - 13 min
1.784.581.959 mkv.12.gz - 80 sec
1.784.439.182 mkv.m8.zcm093 - 9 min
1.784.355.455 mkv.ect.10007.zip - 105 min
1.784.044.948 mkv.b2047.bcm130 - 10 min

It's an interesting data point, but I don't think comparing tools on uncompressible data is the normal use case.

The "unstable" version of the Squash benchmark contains libdeflate, also against various other Deflate RFC compliant implementations: https://quixdb.github.io/squash-benchmark/unstable/

[Zyzzyva]

For what it's worth, we added this to the Samtools/Htslib packages for manipulating DNA sequencing files: https://github.com/samtools/htslib/pull/581

The speed gains were very much worth it. We also fuzz tested with American Fuzzy Lop and found no issues. A big thumbs up from me.

The only thing I can see lacking, which sadly I didn't get around to fixing, is a proper "make install" target.

I've added "make install" and "make uninstall" targets.