Assessing compression library reliability

[dbbd]

At my workplace we need to use a compression algorithm for some project. The requirements are same as any - compression performance vs. cpu utilization, suitable api and so on.
But now I was asked to assess the reliability of available libraries.

One option is looking at commits - snappy for example has no commits in the past 2 years, so I can consider it stable and reliable. LZ4 on the other hand has recent bug fixes,
and that to some people means it is still not reliable enough.
Another option is looking at how many and who is using each library. But the data is sketchy, and many projects support/use multiple libraries.

I'm looking for more ideas about how to assess libraries' reliability as a factor in selecting the best fit. If I could "prove" that they are all reliable enough for use in a product, I can move on to the more technical aspects in making the decision.

[joerg]

@dbbd: welcome in the encode-forum

i think - first decide

what you have

-- low cpu or high cpu ressources or GPU-ressources

and what you want

-- compression fast or very good ?
-- decompression must be very fast (= often decompressing?) or for archiv-purposes (= decompressing not often)

and then look further


out of the box i propose you to look and test:

http://www.zlib.net/

- stable, patentfree and "very often used in projects"

http://www.oberhumer.com/opensource/lzo/

- stable, patentfree (GPL v2+) - long tested - small code - small cpu requirements
- very portable written in ANSI C
- fast compression and *extremely* fast decompression.

http://www.7-zip.org/

- stable, patentfree - 7z - very good compression - LZMA2 - SDK - higher cpu requirements

http://libbsc.com/

- stable
- but need further testing and modification for compiling - not easy ..
- very innovativ - especially if you have GPU-resources very very fast ...
- i think especially compression modes st5 and st6 are interesting

best regards

[Cyan]

xxxxxx for example has no commits in the past 2 years, so I can consider it stable and reliable

Having no commits looks like a dangerous proxy to judge reliability.
Many open source projects are not updated for extended period simply because they are dead.
It doesn't mean they are reliable.

[just a worm]

I agree with Cyan that there are many dead projects (especially open source projects). But beeing dead isn't necessarily a clear sign of beeing reliable or beeing not reliable. On the one hand it's stable and on the other hand it will become incompatible with new versions of operating systems. But, I think that the availability of the source code (or a detailed documentation how it works) and the terms to use it also affects the reliability in the sence that you can do slight changes yourself if the original autor gives up his work. It could also be ported to a different/new architecture in the future.

[nemequ]

But now I was asked to assess the reliability of available libraries.

One option is looking at commits - snappy for example has no commits in the past 2 years, so I can consider it stable and reliable.

Like Cyan mentioned, this doesn't necessarily mean it is reliable. Based on a message posted to the Snappy mailing list (which I feel a lot of people missed), it seems more that the reason Snappy hasn't seen any recent commits is that it is hard for google to maintain the open-source version of Snappy.

I'm looking for more ideas about how to assess libraries' reliability as a factor in selecting the best fit. If I could "prove" that they are all reliable enough for use in a product, I can move on to the more technical aspects in making the decision.

I think the only way you're going to prove that something is reliable enough for you is to test it. I've caught lots of bugs in the unit tests for Squash (as well as a few from static analyzers and fuzzers) and running the Squash Benchmark, and I actually feel fairly comfortable with all the plugins which aren't currently disabled (zling and doboz), though I'll feel much better after I get around to fuzzing them all. IMO the best you can do is just throw everything you can at the library you're interested in and see if it works. Run it in valgrind, asan, tsan, ubsan, etc. Fuzz it, feed it random data, tiny buffers, huge buffers, truncated data, etc.

[JamesB]

You can try American Fuzzy Lop too (http://lcamtuf.coredump.cx/afl/) to mutate input data and slowly trawl through as many code paths as possible, looking for bugs.

You may need to disable CRC and similar checks though to help it identify problems elsewhere.