MONSTER OF COMPRESSION - New Benchmark -

[Nania Francesco]

COMPETITION:
This pages ranks lossless data compression programs by the compressed size of the 18 types of file for 100.042.152 bytes:
BMP,DIC,DLL,DOC,EXE,HLP,JPG,LOG,MP3,PDF,PNG,PSD,SA VE,TAR,TXT,VOB,WAV and XLS.

RULES:
1) the programs cannot use more than 1 GB of memory;
2) the slow pograms too much in compression (them in fact you cannot overcome 100 times the time employed by the fastest of the programs in to compress the data) you will be classified only to the statistic goals but they won't participate in the definitive classification in how much disqualified;
3) dictionaries cannot be used external punishment the disqualification;
4) if found errors in decompression the program is not classified!

LINK:
http://www.winturtle.netsons.org/MOC/MOC.mht

[Nania Francesco]

Corrected 100 times in 200 times!
Sorry!

[Bulat Ziganshin]

1. it's better to use htm files - afaik, mht may be displayed only by Internet Explorer
2. numbers are hard to read - how about using some other font? or may be, bold attribute should not be used
3. lprepaq should be best here
4. 7z -mx9 uses 691 mb of memory

[Nania Francesco]

For 7Zip used
8 MB DICT
16 MB SOLID BLOCK

[Bulat Ziganshin]

it's not ultra setting. use -mx9 if you want to imrove 7zip compression ratio

[Hahobas]

Can't see your website, just the code shows up. I'm running firefox on linux.

[Black_Fox1]

No need for saving in MHT, when there's just one text-only file
Nice idea, Francesco!

[Matt Mahoney]

Cant see your website, just the code shows up. Im running firefox on linux.

Same thing using Firefox in WinXP.

[maadjordan]

here is the unpacked form (html.) its 9KB

http://rapidshare.com/files/66422733/MOC.7z.html

[squxe]

mht may be displayed only by Internet Explorer

Yes, I use Opera and cant see anything

[Uwe Herklotz]

Quoting http://www.winturtle.netsons.org/MOC/MOC.mht:
"FINALITY: Find The Best Pratical Compressor!"

* There is not much sense to test ultra/best settings if looking for best practical use.
* Decompression time is at least as important as compression time.
* There are many other features determining practical usage of a compressor/archiver. For most users things like user interface, portability/stability, security, ... are more important.

Finally, if you plan just another compression-ratio-driven benchmark then IMHO there is no need for it. If you realy want to go for "Best Pratical Compressor" then consider what this means for the average user.

[Gish]

COMPETITION:
This pages ranks lossless data compression programs by the compressed size of the 18 types of file for 100.042.152 bytes

I like the idea, I was thinking to such a test (just with a lower RAM limit) but to compare program using the same amount of RAM.

Its important to define a test that anyone can repeat:
- The table should list the parameters used by each program
- Test-set should be released: why not using a public set of file, e.g. from sourceforge? They also have fast download.

For instance, I was thinking to these tests:
TEST1: This test is done on the star catalog stars_5_1v0_0.cat (a single binary file, 46,096,018 bytes, containing data of 7,6M stars) thus it represent a real set of data in binary format that resemble "random data".
This test should show behavior of compression on a set of binary data that havent a known distribution (apparently random).
The file can be downloaded here:
http://sourceforge.net/project/showfiles.php?group _id=48857

TEST2: This test is done on Filezilla (files extracted form FileZilla_3.0.2.1_win32.zip, packed in a uncompressed tar file, 10,621,952 bytes). This is a Open Source program composed by a unique large exe files (6.3Mbytes, not compressed or packed) and other 250 very small files.
This test should check behavior in compressing large executable code (not already packed with UPX or similar) in a real situation, with additional small resource files (small binary files), difficult to compress.
The files can be downloaded here:
http://sourceforge.net/project/showfiles.php?group _id=21558

TEST3: This is done on Scourge, a huge game with lot of graphic and audio resources (scourge-0.19.zip, packed in a uncompressed tar file, 163,835,904 bytes).
This last test should check the ability to compress graphic and audio with different (possibly not compressed) formats : BMP, WAV, OGG, 3DS.
The executables, as in TEST2, are not packed/compressed; apart for OGG, the other formats does not use a built-in compression.
The file can be downloaded here:
http://sourceforge.net/project/showfiles.php?group _id=98006

Other TESTS can be done, using other Files on SourceForge, for instance vegastrike_beta_0.5.0.zip, which is >800Mbytes uncompressed with lot of audio/graphic resources (mostly compressed, JPG, PNG, OGG). That test should be harder than TEST3 (above). Or, for instance, the TEST1 can be done with stars_6_2v0_0.cat, which is 150Mb and can be downloaded from the same sourceforge page.
The important thing is to allow anyone to repeat the results, providing all the information. This should allow other users (or even the program authors, if they are interested) to make the tests for you...
Also, there are archivers (of the same authors) that are better than thos you picked (e.g. PIMPLE2 could be better than TC)

Ive already done the above tests (TEST1, TEST2, TEST3, but Ill rpeat them because I missed some data for errors in the scripts) and I found interesting results on the last releases (also not public) of these archivers:

(name)_(release)_(params)_(license)_(Author)_(Rele ase Date)
================================================== ===
(7-zip)_(4.56b)_(-t7z -mx6)_(GPL)_(Igor Pavlov)_(24-Oct-07)
(ccm)_(1.26b)_(4)_(Free ???)_(Christian Martelock)_(29-Oct-07)
(ccmx)_(1.26b)_(4)_(Free ???)_(Christian Martelock)_(29-Oct-07)
(uharc)_(0.60b)_(-mx)_(Free Pers)_(Uwe Herklotz)_(1-Oct-05)
(uharc)_(0.60b)_(-mx -md3276_(Free Pers)_(Uwe Herklotz)_(1-Oct-05)
vuharc)_(0.60b)_(-mx -md32768 -mm+)_(Free Pers)_(Uwe Herklotz)_(1-Oct-05)
(freearc)_(0.40pr1)_(-mx)_(Free ???)_(Bulat Ziganshin)_(24-Oct-07)
(freearc)_(0.40pr1)_(-m5)_(Free ???)_(Bulat Ziganshin)_(24-Oct-07)
(lpaq1)_(1v2)_(6)_(GPL)_(Matt Mahoney)_(19-Sep-07)
(lpaq6)_(6)_(6)_(GPL)_(Matt Mahoney, Alexander Rhatushnyak)_(22-Oct-07)
(lprepaq)_(1.2)_(6)_(GPL ???)_(Christian Schneider, Matt Mahoney)_(1-Oct-07)
(ppmd)_(J)_(-m256 -o16)_(Free Pers)_(Dmitry Shkarin)_(10-May-06)
(ppmd)_(J)_(-m256 -o16)_(Free Pers)_(Dmitry Shkarin)_(10-May-06)
(ppmonstr)_(J)_(-m256 )_(Free Pers)_(Dmitry Shkarin)_(10-May-06)
(ppmonstr)_(J)_(-m256 -o16)_(Free Pers)_(Dmitry Shkarin)_(10-May-06)
(durlica light)_(0.5)_(-m256 -t1)_(Free Pers)_(Dmitry Shkarin)_(1-Apr-06)
(durlica light)_(0.5)_(-m256 -t1 -o16)_(Free Pers)_(Dmitry Shkarin)_(1-Apr-06)
(uda)_(0.301s)_(Free ???)_(dwing)_(19-Dec-06)
(slim)_(0.23d)_(-m256)_(Free ???)_(Serge Voskoboynikov)_(21-Sep-04)
(pimple2)_(nd)_(none)_(Free ???)_(Ilia Muraviev)_(10-Jun-07)
(sr2)_(nd)_(none)_(GPL)_(Matt Mahoney)_(3-Aug-07)
(sr3)_(nd)_(none)_(GPL)_(Matt Mahoney, Nania Francesco Antonio)_(28-Oct-07)


Finally, I could not see the table in IE7, Opera, Firefox. I had to download it and open it with IE to see it.
Maybe you can change the table in a normal JPG or GIF... than anyone will see it.

[Nania Francesco]

Ok is all right, tell me what the format is in which desire to see the page of "MOC"
- HTML
- PHP
- or?
they are to your disposition to compile the Web page;
the Web page is anchors in phase of construction, I will also make a will 7ZIP with compression PPMD!

[Gish]

Quoting http://www.winturtle.netsons.org/MOC/MOC.mht:
"FINALITY: Find The Best Pratical Compressor!"

* There is not much sense to test ultra/best settings if looking for best practical use.
* Decompression time is at least as important as compression time.
* There are many other features determining practical usage of a compressor/archiver. For most users things like user interface, portability/stability, security, ... are more important.

Uwe, I agree with you about all your points. I have a 512Mb PC (which is still quite common configuration) and when Ive done my tests Ive tried to use no more than 256Mb, to avoid memory swap to disk (becaiuse in that case the compression becomes 10x slower).

However... even test with heavy memory usage can be interesting...
Uwe, are you the author of Uharc, arent you?
Ive done the TEST1 , a very difficult sets of nearly random data, (see my previous post) and Ive discovered that Uharc behaves differently (I mean better) than all the other archivers (in the above post). Ive also decompressed the archiver and the binary comparison with the original file was OK.

Have you abandoned the development of UHARC ?

[Gish]

Ok is all right, tell me what the format is in which desire to see the page of "MOC"
- HTML
- PHP
- or?
they are to your disposition to compile the Web page;
the Web page is anchors in phase of construction, I will also make a will 7ZIP with compression PPMD!

I suppose it would be fine if you can do a "plain" html page, in that way all the Firefox/Opera/IE users will be able to see it!

Remember to put the compression parameters in the table.
Also, there are archivers (FreeArc, 7-zip, Slim) that do not need to work on a single (TAR) file: they will give better results on the set of files, and you should have a column in you table that tells if the compression is applied on a single TAR file or not (as in the tables of maximumcompression and compressionmax web sites)

Francesco, what is the best among your compressors? Turtle, Rings, Hook, or LZC? (Id like to make a test with it, too).

[LovePimple]

1. its better to use htm files - afaik, mht may be displayed only by Internet Explorer

I agree! HTML is best.

[Nania Francesco]

To the moment have added anything and have written him in HTML! Be all right!

[nimdamsk]

My 2 cents.
Id prefer fully configurable dynamic (i.e. php/mysql) page with algorithm, author, country, year, etc.. grouping, with ability to compare only selected archivers etc. And IMHO it would be better to model different user groups - programmers, artisits, gamers, housewifes As its already done in other benchmarks like WinMark.

Ive done the TEST1 , a very difficult sets of nearly random data, (see my previous post) and Ive discovered that Uharc behaves differently (I mean better) than all the other archivers (in the above post).

Soon Bulat will tell you right options for FreeArc to beat everyone

[Gish]

Soon Bulat will tell you right options for FreeArc to beat everyone

I thanks ALL the Authors for their programs... but at TEST1 Uharc was the only one to get a result different from all the others while FreeArc was the only one that get an archive larger than the original file ...
However Ill repeat the tests, or maybe its just a bug in the 0.40pre-release of FreeArc...

Stars database of the NOMAD catalog, further details baout those catalogs, used by Stellarium, are here: http://www.stellarium.org/wiki/index.php/Stars

I wonder if that data are random or if there is some "universal law" behind them...

[Gish]

In the table (picture) are the tests on Star Catalogue.
The "TAR" column only means that there is a single file (stars_5_1v0_0.cat) not that I've done a tar file.

What I found interesting is the behaviour of:
UHARC 0.6 whic compress significantly better than others (UDA is better, but it is 20 times slower, because it is based on PAQ8I); also FreeArc and PPMD have strange behaviour and they give an archive larger than the original file(bug? wrong parameters?).

The Table is here [FIXED]: (CLICK TO ENLARGE)


http://img511.imageshack.us/img511/5022/test1compr essionty5.png

[Black_Fox1]

I will also make a will 7ZIP with compression PPMD!

Please, dont. LZMA is the reason for using 7-zip, not PPMD.

Anyway, I think Gish and nimdamsk have a good idea. We need another publicly available large-scale benchmark like LTCB, but more general purposed, that wil have multiple contributors/testers and will be more dynamical...

[nimdamsk]

In the table (picture) are the tests on Star Catalogue.

1. Two ppmd with the same options and different results
2. "Uharc -mx" and "uharc -mx -mm+" gives the same result, you dont need to use "-mm+" as its used by default

[Gish]

1. Two ppmd with the same options and different results

Fixed, the first was without "-o16"

2. "Uharc -mx" and "uharc -mx -mm+" gives the same result, you dont need to use "-mm+" as its used by default

Fixed, thanks, this will spare a couple of tests...

However the link to the picture (PNG) is very slowww...(its a free hosting for pictures, at the moment I could not find other places for it).

Ill try to launch a longer test on the catalogue stars_6_2v0_0.cat (26,615,233 stars, 153M, same location at sourceforge).
It will run longer on more archives and I should have results the next week...

The New Table is here:

URL
http://img511.imageshack.us/img511/5022/test1compr essionty5.png

[Nania Francesco]

MONSTER OF COMPRESSION
NEW LINK:
http://www.winturtle.netsons.org/MOC/MOC.htm

[Simon Berger]

@Gish
I tested some things with uharc. -md64 seems to compress a little bit better and is faster. Also you can disable multimedia detaction (because thats no known file type). But I was surprised by the result. The compression was 10 seconds faster (with 109 seconds without 100seconds) and it got also more compression (with -mm+ 42392298 without 42390536).

Was surely wrong:
-mm+ (or default) | -mm-
109s 100s
42392298byte 42390536

[Nania Francesco]

MONSTER OF COMPRESSION

Added in benchmark
- Frearc 0.36
- quad
- TarsalLZP 21 .08.2007
- cabarc
- Winturtle 1.30
- Thor 0.96a

[Fallon]

However the link to the picture (PNG) is very slowww...(its a free hosting for pictures, at the moment I could not find other places for it).

From my bookmarks, two other image-upload-services, but not tested by me:
http://www.imagevenue.com/
http://www.imagez.org/

A problem with imageshack is that links are disabled when a lot of bandwidth is used.
If you upload popular pictures of paintings by artists like Vincent ... http://www.vangoghgallery.com/catalog/Painting/610 /Thatched-Cottages-by-a-Hill.html
http://www.delftintegraal.tudelft.nl/info/images/M ondriaan.jpg
... links can soon be gone.

[Nania Francesco]

MONSTER OF COMPRESSION

Added in benchmark :
- Ash04a (disqualified for over time);
- LZPM 11
- FPAQ0S6
- SBC
- WINUHA
- M99

[kaitz]

Nice site, but font is to bold.

[Nania Francesco]

Added
- Tornado
- WinRK 3.03