GCC 4.4 and compression speed

[Hahobas]

I'm always keeping tabs on the progress of GCC and I have done some compiles from the latest svn trunk. Since paq is an easy compile (and is Linux compatible), I decided to do a few test with gcc 4.4.0 20090208 (experimental) vs gcc4.3.2. I know paq is much faster using the hand-written assembly, but this test is about the speed of the compiler produced code.

Below are some results. I included the compile line I used, and then the test itself.
-------------------
g++ paq9a.cpp -DUNIX -O2 -s -fomit-frame-pointer -ftree-switch-conversion -floop-interchange -floop-strip-mine -floop-block -o paq9p_gcc44_O2
./paq9p_gcc44_O2 a archive enwik8
416.41 seconds

2nd try, without loop optimization switches
g++ paq9a.cpp -DUNIX -O3 -s -fomit-frame-pointer -ftree-switch-conversion -floop-interchange -floop-strip-mine -floop-block -o paq9p_gcc44_O3
./paq9p_gcc44_O3 a archive enwik8
392.13 seconds

/storage/gcc/gcc_staging_loop/usr/local/bin/g++ paq9a.cpp -DUNIX -O3 -s -fomit-frame-pointer -o paq9p_gcc44_O3
./paq9p_gcc44_O3 a archive /storage/compression-test-files/enwik8
392.66 seconds

g++ paq9a.cpp -DUNIX -O3 -s -fomit-frame-pointer -o paq9p_gcc432_O3
./paq9p_gcc432_O3 a archive enwik8
415.74 sec

g++ paq9a.cpp -DUNIX -O2 -s -fomit-frame-pointer -o paq9p_gcc432_O2
./paq9p_gcc432_O2 a archive enwik8
414.76 sec
-------------------

As you can see in the first tests I tried out the new optimization switches (-ftree-switch-conversion -floop-interchange -floop-strip-mine -floop-block) but it didn't seem to make much of a difference with paq9a. Perhaps other algorithms might get some mileage out of it. What I did find interesting however was the 5.8% performance improvement while using the O3 optimization level. I'd love to be able to try gcc 4.4 with some of my favorite compressors ( Nanozip, CCM, BCM, etc). So if any of you who would like to try it, I'm sure it'll produce some interesting results.

If you want to use the new snazzy loop optimizations, they require two external libraries when you compile gcc. http://gcc.gnu.org/wiki/Graphite_Build was very helpful to me to get it working.

[Shelwien]

Can you test http://ctxmodel.net/files/fpaq0pv4b2.rar ?

[Hahobas]

I'd be happy to test them, but I can't figure out how to compile your source. The .inc files are especially confusing to me. What command lines do you use to compile your executables with gcc? Keep in mind I'm running Linux.

[Shelwien]

I thought its kinda obvious to try to compile the single cpp file,
and anyway the compilation scripts (c.bat) are included.
But in the case if you have problems with __declspec (though
my cygwin gcc just says warning for it) and/or don't want to
deal with perl and preprocessing for -tpl versions,
here's an update:

http://ctxmodel.net/files/fpaq0pv4b3.rar

[Hahobas]

I thought its kinda obvious to try to compile the single cpp file

That's what I tried at first, but I got so many compile errors I assumed I was doing something wrong. This time I looked closer at the errors and it seems the problem is your code is not 64-bit compatible. I would try compiling it into a 32-bit executables, but I only compiled gcc with 64-bit compatibility.

It doesn't look like it will take much to make it more portable though, these are the errors I got:

In file included from fpaq0pv4B.cpp:4:
buff.inc: In function ?byte* anew(int, int)?:
buff.inc:14: error: cast from ?byte*? to ?uint? loses precision
In file included from fpaq0pv4B.cpp:6:
file_std.inc: In function ?uint file_open(char*)?:
file_std.inc:3: error: cast from ?FILE*? to ?uint? loses precision
file_std.inc: In function ?uint file_make(char*)?:
file_std.inc:7: error: cast from ?FILE*? to ?uint? loses precision

I tried changing all the uint to uint32 but it was still complaining about loss of precision.

[Simon Berger]

The problem is, that *File is a 64bit pointer. You would need uint64

[Hahobas]

Hmm, something's not right. I changed it to uint64 and gcc is now saying that uint64 is not a valid type. Any ideas? I'm running Fedora 10 amd_64.

[Shelwien]

test with a printf( "sizeof(FILE*)=%i\n", sizeof(FILE*) ); ?

[Hahobas]

sizeof(FILE*)=8

[Simon Berger]

Yeah sure 8 byte = 64 bit = pointer of 64 bit system

[Hahobas]

I downloaded a Fedora 11 Alpha x86 VM, and was able to compile 32-bit versions fpaq0pv4b2 with gcc4.4 using it and compare it to 32-bit compiles using gcc 4.3.1. I'm not sure what version the gcc4.4 compiler in Fedora 11 Alpha is, or if it is using the new register allocator, but the results look very promising.

I did simple compiles with no fancy switches except the general optimization switch. For example: g++ fpaq0pv4b2.cpp -O1 -o fpaq_gcc44

Test System: AMD Athlon 64 3000+ Venice, 2GIG RAM
Each of these timings is the average of 5 runnings.
enwik8 (100000000 bytes) -> enwik8_compressed (61287637 bytes)

gcc 4.3.1
-O1 compressed in 8.60 s, decompressed in 8.02s
-O2 compressed in 7.81s, decompressed in 7.93s
-O3 compressed in in 7.53 s, decompressed in 7.77s

gcc 4.4 enwik8 (100000000 bytes) -> enwik8_gcc44 (61287637 bytes)
-O1 compressed in 6.85 s (25.5% faster), decompressed in 7.35s (9.12% faster)
-O2 compressed in 6.62 s (18% faster), decompressed in 7.42s (6.9% faster)
-O3 compressed in 6.82 s(10.4% faster), decompressed in 7.69s (1% faster)

[Hahobas]

Shelwien, Check out some benchmarks of your fpaq0pv4b2 compiled with llvm.

Test file was enwik9

gcc_431_O3 - 68.87 seconds
gcc_44_O3 - 61.71 seconds (10.3% faster)
llvm_O3 - 54.76 seconds ( 20.5% faster than gcc4.3.1 and 10.4% faster than gcc 4.4)

I just used the llvm 2.4 binary downloadable from:
http://llvm.org/

I would try clang, but there isn't c++ support yet. I also tried paq9a, but llvm produced poorer performing executables than gcc4.4 in that case.

[Bulat Ziganshin]

i have tried llvm-g++ compiler on mytornado. it produced slower executable but at least it works. all what you need is to download

http://llvm.org/releases/2.4/llvm-gc...ingw32.tar.bz2
http://llvm.org/releases/2.4/llvm-2....ingw32.tar.bz2

decompress them, include their executable directories to the path and make sure that ld.exe is also available in path

then i just have replaced g++ with llvm-g++ in my compile script

[Arkanosis]

Hello,

llvm 2.5 has been released: http://llvm.org/releases/download.html#2.5

Note that it still generates poor floating point code for x86 if SSE2 is not enabled.

Regards,

[jo.henke]

If somebody is interested: I found an issue (bug?) with gcc 4.4 and Tornado.

http://encode.su/forum/showpost.php?...&postcount=143