PAQ8K

[Black_Fox1]

Bill Pettis released new version of PAQ!
http://ilovemyking.googlepages.com/paq8k.zip

EDIT: On brief test it seems, that majority of files will give better results

[LovePimple]

Thanks BF!

Is this version any slower than PAQ8jd?

[Black_Fox1]

I tried only PNG up to now - PAQ8K is 3-4x slower on the one from my testset.

[LovePimple]

PAQ8K is 3-4x slower on the one from my testset.

WOW! That is much slower than I had expected.

[thometal]

yeah iam also testing and waiting........

Why is so slow?
What of the things did you implement, thats so slow?
Did you compile with asm?
What is a Chartmodel?

[thometal]

here my first test:

Cokumente und EinstellungenThomasDesktop>paq8jdsse.exe -6 win.wav
Creating archive win.wav.paq8jd with 1 file(s)...
win.wav 1709576 -> 1404345
1709576 -> 1404375
Time 300.83 sec, used 531535259 bytes of memory

Cokumente und EinstellungenThomasDesktop>paq8k.exe -6 win win.wav
Creating archive win.paq8k with 1 file(s)...
win.wav 1709576 -> 1409989
1709576 -> 1410018
Time 852.45 sec, used 419251344 bytes of memory

Cokumente und EinstellungenThomasDesktop>

I think its not good that you reduce memory requirements because of benchmarking with older versions.

[thometal]

here my second testset

Compressed Tar was the old Transport Tycoon + OpenTTD with only a few save games.

Cokumente und EinstellungenThomasDesktop>paq8jdsse.exe -6 TT.tar
Creating archive TT.tar.paq8jd with 1 file(s)...
TT.tar 37218304 -> 22987894
37218304 -> 22987924
Time 6487.64 sec, used 539930268 bytes of memory

Cokumente und EinstellungenThomasDesktop>paq8k.exe -6 8k TT.tar
Creating archive 8k.paq8k with 1 file(s)...
TT.tar 37218304 -> 22963807
37218304 -> 22963836
Time 16798.89 sec, used 427646354 bytes of memory

[LovePimple]

Thanks thometal!

I had planned to use a slightly modified version of PAQ8k with PeaZip but its far too slow to replace PAQ8jd.

[Matt Mahoney]

I tested paq8k on the Calgary corpus. It compresses about 0.1% smaller than paq8jd but is 3 times slower. It is the top result now.
http://cs.fit.edu/~mmahoney/compression/

On enwik8 it compresses worse than paq8jd. It took about 13 hours. I did not verify decompression and I have no plans to test on enwik9.
paq8jd -7 18,158,159
-8 17,943,042
paq8k -8 18,239,915
http://cs.fit.edu/~mmahoney/compression/text.html# 1334

I looked at the code and I still can't tell what a chart model is. Putting the indirect model in a loop doesn't make sense to me either.

Here are the Calgary tests for paq8jd and paq8k. I had to put all the filenames on the command line in paq8k because wildcards and directory traverse are broken. (Probably both could be fixed by compiling with g++ -DWINDOWS but I used the .exe from the website).

C:
es>timer compresspaq8jdpaq8jd -7 calgary
Timer 3.01 Copyright (c) 2002-2003 Igor Pavlov 2003-07-10
Creating archive calgary.paq8jd with 14 file(s)...
calgary/BIB 111261 -> 20888
calgary/BOOK1 768771 -> 192375
calgary/BOOK2 610856 -> 117535
calgary/GEO 102400 -> 43983
calgary/NEWS 377109 -> 83911
calgary/OBJ1 21504 -> 7366
calgary/OBJ2 246814 -> 44522
calgary/PAPER1 53161 -> 10510
calgary/PAPER2 82199 -> 16572
calgary/PIC 513216 -> 23025
calgary/PROGC 39611 -> 8380
calgary/PROGL 71646 -> 9779
calgary/PROGP 49379 -> 6904
calgary/TRANS 93695 -> 10122
3141622 -> 596179
Time 527.42 sec, used 1030659022 bytes of memory

Kernel Time = 1.609 = 00:00:01.609 = 0%
User Time = 489.968 = 00:08:09.968 = 92%
Process Time = 491.578 = 00:08:11.578 = 93%
Global Time = 527.531 = 00:08:47.531 = 100%


C:
escalgary>timer
escompresspaq8kpaq8k -7 ..cal bib book1 book2 geo news obj1 obj2 paper1 paper2 pic progc progl progp trans

Timer 3.01 Copyright (c) 2002-2003 Igor Pavlov 2003-07-10
Creating archive ..cal.paq8k with 14 file(s)...
bib 111261 -> 20700
book1 768771 -> 193523
book2 610856 -> 117420
geo 102400 -> 43976
news 377109 -> 83522
obj1 21504 -> 7185
obj2 246814 -> 43440
paper1 53161 -> 10526
paper2 82199 -> 16660
pic 513216 -> 23738
progc 39611 -> 8294
progl 71646 -> 9651
progp 49379 -> 6771
trans 93695 -> 9937
3141622 -> 595537
Time 1540.77 sec, used 767380400 bytes of memory

Kernel Time = 1.328 = 00:00:01.328 = 0%
User Time = 1434.500 = 00:23:54.500 = 93%
Process Time = 1435.828 = 00:23:55.828 = 93%
Global Time = 1540.828 = 00:25:40.828 = 100%

[LovePimple]

Thanks for the update Matt!

I tested paq8k on the Calgary corpus. It compresses about 0.1% smaller than paq8jd but is 3 times slower. It is the top result now.

Im sticking with PAQ8jd for the time being!

On enwik8 it compresses worse than paq8jd. It took about 13 hours. I did not verify decompression and I have no plans to test on enwik9.

I dont blame you!

[Black_Fox1]

What of the things did you implement, thats so slow?
Did you compile with asm?
What is a Chartmodel?

I wonder why is Bill so humble with any details... there isnt a word about it even on his site
But it really is top PAQ version now