Snappy 1.0.5

[Sesse]

Hi,

Some of you might be interested: Snappy 1.0.5 is out, as always on http://code.google.com/p/snappy/ . There are some fixes and performance improvements as usual; the most dramatic change is on ARM, with as much as 30?40% faster decompression and 25?45% faster compression for ARMv7 and newer, but there are also smaller changes (3?10% faster decompression) for the primary target of x86-64.

/* Steinar */

[m^2]

Hey Sesse.
I'd like to know what do you think about the C port of Snappy? Maybe you could share a few words?

[Sesse]

Hey Sesse.
I'd like to know what do you think about the C port of Snappy? Maybe you could share a few words?

I don't know if there's that much to say? Andi Kleen wanted to include Snappy in the Linux kernel (which makes sense, given that e.g. LZO is already there and does useful things); given that Linux is C-only (no C++), a C port was naturally needed. I haven't had anything to do with it code-wise, but as far as I understand it was a straight port and should benchmark similarly to the C++ version. (It generally seems to keep the code structure, comments, etc., so I can follow it quite easily, which I can't necessarily say about all the compression code out there. But then again, I know Snappy much better than any other compressor, so it's maybe not so strange.)

We can't use the C port as the primary version, since C lacks some of the primitives we need to plug many different backends into Snappy efficiently (e.g., decompress from more complex structures than plain arrays), but I'm sure Andi will be able to keep up with relevant performance improvements etc. as they come into the C++ version.

Did that answer your question?

/* Steinar */

[m^2]

I've seen a benchmark (badly designed, but it doesn't matter here) where they got very different results:
http://article.gmane.org/gmane.comp....ms.btrfs/15744
Strength is much different, so at least it was retuned.

Also, unlike the C++ version, it uses Linux headers quite intensively, in particular one from asm/, which I except to have some, possibly significant, impact on performance.

I thought that you knew the stuff and could tell some things about it.

[Sesse]

I've seen a benchmark (badly designed, but it doesn't matter here) where they got very different results:
http://article.gmane.org/gmane.comp....ms.btrfs/15744

It sounds weird, but I haven't looked into it. Note that it doesn't mention whether it's run in 32-bit or 64-bit mode (which makes a huge difference from Snappy), and it claims compilation using ?GCC 4.7?, which isn't even released yet, so I'd be a bit wary of these results in general. Also, it doesn't say anything about running with or without assertions; enabling assertions is a common mistake when benchmarking these things. There's a git reference, but as far as I can see it's to a full kernel tree, and I haven't got the time right now to try to fish out where the benchmarks might be in there.

Strength is much different, so at least it was retuned.

I haven't heard of any retuning of the algorithm, but you could always contact Andi Kleen and ask him. In general, that's who you should ask about snappy-c, not me

/* Steinar */

[m^2]

Thanks, I guess you're right, I chose to ask you because you were closer and you were independent from here. In fact, you were the only independent person that I know who had a chance of having *any* opinion about the port.

[Sesse]

Thanks, I guess you're right, I chose to ask you because you were closer and you were independent from here. In fact, you were the only independent person that I know who had a chance of having *any* opinion about the port.

I had the time to do a quick test (I downloaded the latest version off github and ran it against C++ Snappy), and I can't reproduce the difference in compression strength. So either there's a problem with the benchmark, it is running with different settings (or source), or there is some unknown bug in the C port. In any case I'd ask them for benchmark source etc. if you're interested.

/* Steinar */

[m^2]

Thanks for the info. They gave the benchmark sources (git://repo.or.cz/linux-2.6/btrfs-unstable.git dev/compression-squad) together with the results.
I'll take a look when I have time (not before weekend).

[m^2]

Apparently I misunderstood that mail. The git branch didn't contain the benchmark, but a Linux kernel.
BTW, I didn't know it's that big.