PIMPLE2 compression

[encode]

PIMPLE2 is a new data compression engine for the PIM v2 archiver - a further evolution of my CM compression. Its intended to bury PIMPLE v1!

Its at almost written, now Im testing/tuning it - just thinking how much memory/time I can spend on super-compression without going out of bounds of the real world.

Check out some results of the test build version of PIMPLE2. Note that probably I will change the memory usage and other parameters as a result the compression can became slightly higher or lower.

Memory usage: ~140 MB

PIMPLE2 at SFC
A10.jpg: 829,961 bytes
acrord32.exe: 1,179,535 bytes
english.dic: 559,083 bytes
FlashMX.pdf: 3,648,878 bytes
fp.log: 436,647 bytes
mso97.dll: 1,556,940 bytes
ohs.doc: 749,606 bytes
rafale.bmp: 732,003 bytes
vcfiu.hlp: 502,825 bytes
world95.txt: 445,213 bytes
Total: 10,640,691 bytes

Having said that this is the best thing Ive ever made!

PIMPLE2 fits in SFC top ten, outperforming CCM and others!

One thing - its slow. However, its faster than WinUDA or PAQ6...

Watch for updates!

[Black_Fox1]

Great!! That's amazing news, encode Is it going to have about same speed as v1?

[encode]

Depending on settings of PIMPLE v1. If you use PIMPLE v1 with presets PIMPLE2 slower.

Comparison:

MPTRACK.EXE (1,159,172 bytes)

PIMPLE, Extreme preset: 448,771 bytes, 11 sec
PIMPLE2: 418,328 bytes, 23 sec

Two times slower. Anyway, overall, compression is MUCH higher!

[encode]

For example:

reaktor.exe (14,446,592 bytes)

PIMPLE, Extreme preset: 1,858,425 bytes
PIMPLE2: 1,632,466 bytes

[LovePimple]

Its intended to bury PIMPLE v1!

NOW - This is more like it!

[Nania Francesco]

Very Good job!

[Christian]

PIMPLE2 fits in SFC top ten, outperforming CCM and others!

Damn it!! ... Just joking, good job Ilia!

Now, youve just to speed it up some more. But Im sure you will - the algorithm is still very young.

[encode]

But Im sure you will - the algorithm is still very young.

Yepp... However, it belongs to PIMPLE/TC legacy - i.e. I collect all my ideas in one place focusing on compression only. To be honest, compression still can be higher, but speed... - each compression improvement makes PIMPLE2 2X and more times slower. So, at some point my goal is to find a HIGH compression with reasonable time. In addition, I think I will keep memory usage below 200 MB or something - so users with just say 256 MB of RAM can use it. Note that with PIM2/PPMd (var I) I use the same parameters as WinZip 11 make use (order 16, 64 MB, restart model). This is one of the best settings for PPMd - a higher model size can hurt compression on binary data, rebuilding model takes too much time... And so with PIMPLE2, it uses only a few models which keeps statistics in very compact way and with higher amounts of memory usage we even can loose the copression. Anyway, Im under experiments...

[squxe]

Total: 10,640,691 bytes

Great!
9th plase in SFC Top 10! (it beats CCM!)
Wait for release!

[encode]

Testing... But I'm unhappy with speed. Even TC is much faster, especially with highly redundant data.

What I've done with PIMPLE2 is I removed any LZ layers as a result I loose speed and gain compression - adding a new Match model.

In addition, PIMPLE2 has a new Sparse model - which gives very nice compression gain and 2X speed loss.

Again, I can't understand how Chris make his CCM so fast? If it uses many models, etc.

Chris, maybe you share with us about your secret?

I compared PIMPLE2 to PAQ1 - and PAQ1 is faster!

Anyway, I have some ideas to check...

[Matt Mahoney]

How is PIMPLE2? I see it benchmarked on maximumcompression.com but no release?

[encode]

It was released some time ago and available here:
pimple2.zip

[encode]

See:
http://www.encode.su/forums/index.php?action=vthre ad&forum=1&topic=458

[Matt Mahoney]

OK, thanks. I didn't see the original link.