comprox/comprolz updates

[RichSelian]

comprox verion 0.9.0 is here: http://comprox.googlecode.com/files/...x-0.9.0.tar.gz
comprolz version 0.2.0 is here: http://comprox.googlecode.com/files/...z-0.2.0.tar.gz

a simple test for enwik8 (with option "-b100 -f"):

comprox  100000000 => 24281529
comprolz 100000000 => 24280609

[Matt Mahoney]

Updated results. http://mattmahoney.net/dc/text.html

[Matt Mahoney]

I added comprox 0.9.0 and comprolz 0.2.0 to the Silesia corpus.
Comprox crashes when compressing samba with -f.
http://mattmahoney.net/dc/silesia.html

[RichSelian]

I added comprox 0.9.0 and comprolz 0.2.0 to the Silesia corpus.
Comprox crashes when compressing samba with -f.
http://mattmahoney.net/dc/silesia.html

Thanks very much for benchmark and bug report. a bugfix version is here: http://comprox.googlecode.com/files/...bugfix1.tar.gz

[Matt Mahoney]

Thanks. I updated the Silesia benchmark.

[Stephan Busch]

Comprox 0.90 bugfix still crashes on some of my .tar files.
The commandline used were 'comprox -b160 -f e app.tar a.comprox' and 'comprox -m100 -b160 -f e app.tar a.comprox'
Using a smaller blocksize didn't help.

[RichSelian]

Comprox 0.90 bugfix still crashes on some of my .tar files.
The commandline used were 'comprox -b160 -f e app.tar a.comprox' and 'comprox -m100 -b160 -f e app.tar a.comprox'
Using a smaller blocksize didn't help.

Thanks for reporting. Could you upload app.tar or paste a download link for me? It will help a lot with the problem.

[Stephan Busch]

Yep. Here you go: http://www.squeezechart.com/app.tar

[RichSelian]

Yep. Here you go: http://www.squeezechart.com/app.tar

Thank you very much, the bug is fixed.
New version is here:
http://comprox.googlecode.com/files/...x-0.9.1.tar.gz
http://comprox.googlecode.com/files/...z-0.2.1.tar.gz

[Stephan Busch]

This version of COMPROX still can't compress my app.tar.
The source was compiled using MinGW 4.6.2 and this commandline: gcc -O3 -march=native -s *.c

The executable always crashes as soon as the LZ77 encoding wants to start.

[RichSelian]

This version of COMPROX still can't compress my app.tar.
The source was compiled using MinGW 4.6.2 and this commandline: gcc -O3 -march=native -s *.c

The executable always crashes as soon as the LZ77 encoding wants to start.

That's very strange, I tested this version with valgrind under linux, compressing app.tar for hours but without an error. Maybe it's a problem of windows threading? (I defined pthread_create() to CreateThread() on windows.)
I also tested with mingw32-4.7.0 and WINE on linux. it runs as good as native linux executables. But I don't have a windows computer, so I'm not able to locate the bug and fix it.

[Matt Mahoney]

Same here. comprox -b80 e app.tar app.tar.cpx crashes. Also -b96, -b128, -b160, but -b64 is OK. Compiled with gcc 4.7.0 -O3 -s -march=native (also just -O3) in 32 bit Windows Vista.

[RichSelian]

Same here. comprox -b80 e app.tar app.tar.cpx crashes. Also -b96, -b128, -b160, but -b64 is OK. Compiled with gcc 4.7.0 -O3 -s -march=native (also just -O3) in 32 bit Windows Vista.

I fix a bug that may cause memory access violation, but I'm not sure that's the reason causing windows version crash. give it a try, please.

[Stephan Busch]

This bugfix version seems to work. I compiled using 'gcc -O3 -march=native -s *.c'; the executable is 91.648 Bytes in size.
Thank you Rich.

[RichSelian]

0.10.0 is here: http://comprox.googlecode.com/files/...-0.10.0.tar.gz

I made comprox and comprolz together in a package since they share many common codes. This version should have chance compressing enwik9 to less than 200MB!

[Nania Francesco]

Comprox v. 0.10.0 tested in WCC!
Usually I always congratulate them all when they start and this time I will respond differently because I think you can do much better seen achievements in bmp, text,dat,xml and exe.

Great work but you have to improve speed of decompression and maybe try and better than with a traditional arithmetic coding with the contexts for the bits and bytes to not!

Overall great improvement!

[Nania Francesco]

ComproLZ v. 0.10.0 tested in WCC. It is slower than comprox!

[Matt Mahoney]

Updated LTCB. Compression is much improved. http://mattmahoney.net/dc/text.html#1987

[RichSelian]

version 0.11.0 is here: http://comprox.googlecode.com/files/...-0.11.0.tar.gz

Now the dictionary is generated from input file and no more static dictionary file is needed.

[Matt Mahoney]

Compression is worse on LTCB but better on Silesia corpus.
http://mattmahoney.net/dc/text.html#1987
http://mattmahoney.net/dc/silesia.html (but 0.7.0 is still tops).

I guess that is because previous static dictionary was derived from LTCB. Also comprox -m100 it took 30 times longer to compress enwik9 than enwik8. I guess that is due to growing hash chains. With default -m40, it is 12 times longer.

Also comprox -b250 enwik9 crashes (but enwik8 is OK). I had to use -b200. comprolz -b250 was OK.

Nice to have .exe included so I don't have to compile.

[RichSelian]

Compression is worse on LTCB but better on Silesia corpus.
http://mattmahoney.net/dc/text.html#1987
http://mattmahoney.net/dc/silesia.html (but 0.7.0 is still tops).

I guess that is because previous static dictionary was derived from LTCB. Also comprox -m100 it took 30 times longer to compress enwik9 than enwik8. I guess that is due to growing hash chains. With default -m40, it is 12 times longer.

Also comprox -b250 enwik9 crashes (but enwik8 is OK). I had to use -b200. comprolz -b250 was OK.

Nice to have .exe included so I don't have to compile.

a bugfix version: http://comprox.googlecode.com/files/...bugfix1.tar.gz
v0.11.0 should compress enwik8 to about 23070KB with default options. but due to an "undefined behavior" in the code, the win32 version gave a worse result. now the bug is fixed

[Matt Mahoney]

The bug fix gives a big improvement on LTCB (moves up in rankings) and a small improvement on the Silesia corpus.