NEW MOC BENCHMARK

[Nania Francesco]

MONSTER OF COMPRESSION
BENCHMARK

By Nania Francesco Antonio.

This benchmark ranks lossless data compression programs by the compressed size for 2.046.886.013 bytes:

RULES:
1) No external dictionaries;
2) No Memory limit;
3) No errors!
4) Solid Mode only for archivers!
http://heartofcomp.altervista.org/MOC/MOC.htm

[Matt Mahoney]

Also no slow programs like paq I guess

[Nania Francesco]

I have decided to make a will all the programs understood paq, zpaq etc!

Hi Matt!

[Alexander Rhatushnyak]

BMP IMAGES [CONVERTED TO BMP 24B FORMAT]
http://www.bigfoto.com/sculpture-photo.jpg
http://www.bigfoto.com/sites/main/ae...witzerland.JPG
http://www.ianskipworth.com/suig/ima...per/005xga.jpg
http://www.freewallpapers.to/animals/tiger1b.htm
http://www.freewallpapers.to/cars/merc11.htm
http://digilander.libero.it/iono1/fa...wallpapers.jpg
http://fantasyartdesign.com/free-wal...hip-poster.jpg
http://www.nasa.gov/images/content/1...re11Nov_lg.jpg
http://www.nasa.gov/images/content/1...o%20RGB_Hi.jpg
http://bandannie.files.wordpress.com/2009/03/kepler.jpg
http://apod.nasa.gov/apod/image/0812...e_nasa_big.jpg
http://www.phys.ncku.edu.tw/~astrolab/mirrors/apod_e/image/0907/apollo11return_nasa_big.jpg
http://www.lgblog.it/wp-content/uplo...sts122_big.jpg
http://warisboring.com/wp-content/up...ary_robots.jpg
http://marsprogram.jpl.nasa.gov/MPF/...terSResRGB.jpg
http://z.about.com/d/gardening/1/0/U..._Twilight2.jpg
http://peach.blender.org/wp-content/...buck_bunny.jpg
http://www.trekearth.com/gallery/Eur...oto1155597.htm
http://cinematicpassions.files.wordp...nice-italy.jpg

All bmp images in the test set are ex-JPEGs, there is no exception.
It's very hard to find uncompressed big images on the web, isn't it?
And there is still no special model for decompressed JPEGs in paq8*, is there?

[Piotr Tarsa]

Some postprocessing to those BMP's should be applied. I mean something like that: http://www.cs.tut.fi/~foi/SA-DCT/

[m^2]

All bmp images in the test set are ex-JPEGs, there is no exception.
It's very hard to find uncompressed big images on the web, isn't it?
And there is still no special model for decompressed JPEGs in paq8*, is there?

Yeah, I think this part is really flawed.
IMO it would be OK if one of many files would be a JPEG, but all of them?
Pointless.

[Piotr Tarsa]

Here are some PNG samples:
http://r0k.us/graphics/kodak/

[Nania Francesco]

New MOC update is online
added:
- CSC32
- CSC32a2
- CSC32a3
- RZM 0.07H
- YZX v.0.03
- BSC 1.03 (2.x in the next release)

http://heartofcomp.altervista.org/MOC/MOCA.htm

[Nania Francesco]

Corrected more error's in compression/decompression efficiency ratings!

[Fu Siyuan]

Hi Nania, Can you write down the weighted size of each kinds of files in corpus like in Comp.Ratings. I can't download all of the test files but I want to know how the size of each part.

[Nania Francesco]

the question in my language is strange however if have understood well this is the MOC composition!

176 Files

AUDIO ->168.426.586
VIDEO-> 212.669.132
BIN-> 211.709.376
TXT-> 223.596.523
XML-> 61.590.668
BMP-> 214.285.240
DAT->65.006.080
DLL-> 23.313.880
EXE-> 21.501.494
ISO-> 573.026.304
JPG-> 16.736.333
PDF-> 15.087.979
GAME-> 61.742.498
TOOLS->178.045.440

[Raymond_NGhM]

Hi Nania Francesco...
try to add mem.model 4 for CCM & CCMX

[Nania Francesco]

New MOC update is online
added:
- LZTurbo 0.95
- CSC32a4
- Thor 0.96
- YZX v.0.04
- BSC 2.1
http://heartofcomp.altervista.org/MOC/MOCA.htm

[danimor82]

Nania Francesco can you update Nanozip 0.08 alpha? Thanks a lot.

[Lasse Reinhold]

Where is qpress?

[joerg]

can you please test the new bsc 2.2.0 with the -m2 switch (ST5/Schindler Transformation order 5) too ?

[Nania Francesco]

New MOC update is online
added:

- NANOZIP 0.8A
- CSC32A6
- BSC 2.2.0

[Nania Francesco]

New MOC update is online
added:
- lzham
- csc32 final
- crush
- 7zip 9.20

[joerg]

@nania: thank you for the new release of the moc - benchmark

if you can, please test in the next release too

a) the actual version of bsc from http://libbsc.com/Documents/bsc-2.6.1-x86.zip

with

method st5: bsc261 e inputfile outputfile -m5 -cp -b328 -t -M128
or bsc261 e inputfile outputfile -m5 -cp -b328 -t -M192

method st6: bsc261 e inputfile outputfile -m6 -cp -b328 -t -M128
or bsc261 e inputfile outputfile -m6 -cp -b328 -t -M192

and
b) the actual version of zp from http://mattmahoney.net/dc/zp.103.zip

with

zp103 -m4 -t2 -b128 inputfile
or
zp103 -m4 -t2 -b512 inputfile

Do you plan to test the new winrar 4 too ?

Do you plan further development of your wonderful "rings 1.3" / "rings 1.5c" ?

In my tests they are fast with a good compression.

Does you even test these versions within your actual benchmark?

best regards

[Nania Francesco]

@joerg

After a year that I have taken as pause sure I try to test many programs and between those which you have name! Rings will become a archiver, is working more like formulation of the program that like realization!
between least a new release of a my program Flashzip!
Hi!

[Nania Francesco]

New Moc Update is online

Added:
- ZP v.1.03
- LPAQ9
- FP8 (New best ... ops ... beast of MOC)

[joerg]

@nania: thank you for the test

but - i think zp can compress better, if we give a little bit more memory
for example:

zp103 -m4 -t2 -b128 inputfile
or
zp103 -m4 -t1 -b128 inputfile

can you please test this ?

best regards

[Matt Mahoney]

Actually zp103 -m4 -b2000 because memory does not depend on block size for -m3 and -m4. zp does not have a solid mode so it might compress better as a tar file.

[Nania Francesco]

Best settings for ZP (max 2GB of memory) ?
Tar mode .. in the future !

[Matt Mahoney]

Memory usage for zp:
-m1 and -m2 use 5x block size per thread after rounding up block size to a power of 2. So -m1 -b32 -t2 would use 320 MB.
-m3 uses 111 MB per thread regardless of block size.
-m4 uses 246 MB per thread regardless of block size.
Number of threads defaults to number of cores. Change with -t.

Block size defaults to -b32 for all compression modes. Maximum is -b2047. Smaller blocks always make compression worse but save memory for -m1 and -m2. The only reason for having blocks for -m3 and -m4 is to allow multithreading. To use all cores use a block size equal to total size divided by number of cores.

Also, I messed up on version numbers. zp 1.03 is newer than zp 2.00. zp 1.03 is derived from pzpaq. Old zp 2.00 is not multithreaded.

So for best compression: zp -m4 -b2047

[joerg]

@Matt Mahoney:
***
For best speed,
the block size should be the input size divided by the number of cores.
For best compression, use one block.
***

what about this?
- better compression with:

zp -m4 -t1 -b2047 inputfile

[Alexander Rhatushnyak]

Why is LPAQ9m not tested with options 8 and 9 ?
The algorithm is CM. And there are two authors, BTW.

It's likely that LPAQ8 will compress better than LPAQ9m,
as the latter is tuned for enwik9 and should be used with DRT preprocessor.

All uncompressed images in the test set are ex-JPEGs, aren't they?
Do you plan to add some undistorted images?

[Nania Francesco]

Thank Matt
- I correct the errors in the next release.
- I single text l' last version of the program
- DRT is an external preprocessor that creates an external dictionary!
- bmp or pgmthey come from JPEG, does not seem to me that it changes the result very! - Alexander I admire you a lot, because does not create yours archiver, even LZ77 based?



Best regards!

[Matt Mahoney]

@Matt Mahoney:
***
For best speed,
the block size should be the input size divided by the number of cores.
For best compression, use one block.
***

what about this?
- better compression with:

zp -m4 -t1 -b2047 inputfile

-t1 would only compress slower without any effect on compression. The reason for having it would be to reduce memory when using BWT modes -m1 or -m2 with large blocks. -m4 uses 246 MB per thread so probably the default -t would work. Also, -t would only have an effect if you compress more than one file at a time or if the file is larger than 2 GB because otherwise you only get 1 thread anyway.

[Alexander Rhatushnyak]

Alexander I admire you a lot, because does not create yours archiver, even LZ77 based?

sorry, I didn't understand this question.

- bmp or pgmthey come from JPEG, does not seem to me that it changes the result very!

Can you see squares in the attached image?