Metacompressor.com benchmark updates

[Sportman]

Metacompressor.com was started as big files speed benchmark, to offer more diversity I added today an option to add your own test files. Maximal 3 each user and max 10MB zipped.

First login and there click "add test file":
http://www.metacompressor.com/login.aspx

New users can register here:
http://www.metacompressor.com/newuser.aspx

Archivers can be uploaded here:
http://www.metacompressor.com/submit.aspx

Results can be watched here:
http://www.metacompressor.com/uploads.aspx

Code: LongNightsDebugging

[osmanturan]

Thanks sportman! Step by step, you are building a huge benchmark portal

[Nania Francesco]

Surely a test very useful and complete! Very Good job!

[Sportman]

After selecting a test file or uploading a test file it's now possible to select an archiver from a list of archivers instead of uploading for each test an archiver:
http://www.metacompressor.com/submit.aspx

Code: LongNightsDebugging

[osmanturan]

That's what I need! Thanks!

[Nania Francesco]

Good change!

[Sportman]

After selecting an archiver it's now possible to select command line parameters from a list, manual typing is also still possible:
http://www.metacompressor.com/submit.aspx

Code: LongNightsDebugging

[Sportman]

I raised the upload limit for zipped archivers from 1MB to 2MB and the upload limit for zipped test files from 10MB to 15MB.

[pat357]

It seems like "metacompressor.com" is broken : after 6 days, my request is still "waiting" and doesn't show up in the list.
After logging in, I still can see it, but no results yet
Any clues ?

[Sportman]

I unfroze the server by taking it away from the Iceland bank deposit safe to a hotter and saver climate.

No the opposite was true, some archivers where uploaded who started a paq8ox variant in the background that run for ever because paq8ox variants try to make a temp file in root C:\ and that's not allowed under Windows Vista and Windows 2008 Server and fail.

I stopped this endless paq8ox loop and the queued archiver tests one by one finished. I didn't saw queued tests older then one day, maybe the one Pat wait for is a failed one. To make it more clear, I added a status description to the archiver test results displayed after login.

I also added a new option that uploaded new archivers can be stored (if enabled) so others can test with it too.

[Rugxulo]

No the opposite was true, some archivers where uploaded who started a paq8ox variant in the background that run for ever because paq8ox variants try to make a temp file in root C:\ and that's not allowed under Windows Vista and Windows 2008 Server and fail.

Vista etc. don't let you run as Admin by default, and for whatever reason, MSVCRT's tmpfile() tries writing to the root dir by default (doh!). And MinGW (at least current stable, last I checked) still uses such, which is why most paq8ox variants fail. However, if you recompile with Cygwin, OpenWatcom, DJGPP, etc. it will work okay. (That's what I did for paq8o8z.) Of course, the DOS ones won't run under x86-64, but that's (mostly?) AMD's fault. (I prefer DJGPP because it has G++ 4.3.2, which is better than 3.4.x.) :-P

[Sportman]

Thanks, I tested paq8o8zw and it work fine under Windows 2008.

I noticed a strange problem, some archivers like CMM4 stop using CPU after 5-20 seconds and the captured output text also stop showing new data but the process run forever. When I run it manual at the DOS prompt it work fine, it's if the archiver has a problem with being captured by the windows test control application.

[osmanturan]

@Sportman:
Seems you are not using "<machineKey>" tag in your web.config. If you don't use it, every time you will be faced with this error: "Validation of viewstate MAC failed". If you don't know how can you fix this, I can send you a validation key via mail or PM.

[toffer]

@Sportman
Do you use Vista? I used MinGW to compile CMM4, maybe that could be a problem - btw i never had problems like that?! Does it still output something (into the compressed file)?

[osmanturan]

@toffer:
As you know, I use Vista but I didn't get any error when using your compressors. Sportman's machine has Windows 2008 Server x64 which is very similar Vista in architectural point of view. I think, the problem is about server permissions and/or his automated script/program.

[Sportman]

@Sportman:
Seems you are not using "<machineKey>" tag in your web.config. If you don't use it, every time you will be faced with this error: "Validation of viewstate MAC failed".

I generated a machineKey and added it to web.config, let me know if it help.

[osmanturan]

I generated a machineKey and added it to web.config, let me know if it help.

It should help. I faced this error in my web sites 1-2 years ago too. machineKey solved all of related problems. Let me give 20-30 minutes for ensuring whether the problem has been fixed or not. As you know, ASP.NET flushes the session data after this default time out value unless you didn't override a new time out value in web.config.

[osmanturan]

I have tested again. Seems the problem is fixed.

[Sportman]

@Sportman
Do you use Vista? Does it still output something (into the compressed file)?

Windows 2008 64bit what is already mentioned is almost similar as a cleaned Windows Vista.

I did some testing to see when it goes wrong:

file14 - 5,629,573 bytes:
628,073 bytes 3.134 sec. - cmm 54 @infile @outfile cmm d @infile @outfile

file15 - 160,340 bytes:
65,608 bytes 0.256 sec. - cmm 54 @infile @outfile cmm d @infile @outfile

file16 - 7,882,752 bytes:
926,341 bytes 4.49 sec. - cmm 54 @infile @outfile cmm d @infile @outfile

file17 - 1,145,452 bytes:
341,316 bytes 1.024 sec. - cmm 54 @infile @outfile cmm d @infile @outfile

file18 - 8,009,113 bytes:
522,342 bytes 3.929 sec. - cmm 54 @infile @outfile cmm d @infile @outfile

file20 - 7,881,296 bytes:
4,202,005 bytes 6.753 sec. - cmm 54 @infile @outfile cmm d @infile @outfile

file23 - 3,352,576 bytes:
345,557 bytes 1.61 sec. - cmm 54 @infile @outfile cmm d @infile @outfile

file24 - 7,424,946 bytes:
7,220,160 bytes 7.785 sec. - cmm 54 @infile @outfile cmm d @infile @outfile

This tests run fine, when the test file size goes up the problem started:

file13 - 10,485,760 bytes:
After 10.25 seconds CPU time CPU usage go to zero
Output is 9,732,096 bytes and not increasing any more
When I end cmm.exe I see last four rows captured command line output where:
Encoding: 9122/ 9088kB (8.03 bpc). 934 kB/s
Encoding: 9250/ 9216kB (8.03 bpc). 934 kB/s
Encoding: 9379/ 9344kB (8.03 bpc). 935 kB/s
Encoding: 9507/

file19 - 12,305,247 bytes:
After 5.27 seconds CPU time CPU usage go to zero
Output is 573,440 bytes and not increasing any more
When I end cmm.exe I see last four rows captured command line output where:
Encoding: 541/ 9088kB (0.48 bpc). 1831 kB/s
Encoding: 548/ 9216kB (0.48 bpc). 1834 kB/s
Encoding: 555/ 9344kB (0.48 bpc). 1831 kB/s
Encoding: 563/

file27 - 14,357,084 bytes:
After 9.67 seconds CPU time CPU usage go to zero
Output is 7,270,400 bytes and not increasing any more
When I end cmm.exe I see last four rows captured command line output where:
Encoding: 6819/ 9088kB (6.00 bpc). 992 kB/s
Encoding: 6914/ 9216kB (6.00 bpc). 991 kB/s
Encoding: 7008/ 9344kB (6.00 bpc). 991 kB/s
Encoding: 7103/

At all three failed tests the last four command line output rows show 9088kB, 9216kB, 9344kB and freeze just before printing I guess 9344kB + 128kB = 9472kB.

[toffer]

But all of that is still related to your automated testing? Not "manual" running at the command prompt. I might recompile it using a different compiler. ATM i cannot imagine any bugs which cause stuff like that. Let me think a bit about that...

BTW what other archivers suffer from such a behavior? Maybe it's a MinGW related problem?

[Bulat Ziganshin]

my programs are compiled using mingw. freearc, 4x4, tornado. but i don't use tmpfile()

[toffer]

I don't use it, too. Does M1 work for you? If yes, that's one more reason to replace CMM4's I/O system (it should become faster, as a bonus).

@Bulat
Which MinGW version did you use?

[Bulat Ziganshin]

some gcc 3.4.x

[Sportman]

But all of that is still related to your automated testing? Not "manual" running at the command prompt. I might recompile it using a different compiler. ATM i cannot imagine any bugs which cause stuff like that. Let me think a bit about that...

BTW what other archivers suffer from such a behavior? Maybe it's a MinGW related problem?

Yes they have no problem at the command prompt. The automatic test application start every archiver in a new separate thread and this thread poll every second to capture the redirect standard and error output till the archiver is finished.

So far I saw that archivers CMM, CCM and CCMX has this issue. CCM and CCMX have no problem with file13, file19 and file27 but from file11 or bigger they freeze at similar way.

[toffer]

I used GCC 4.2.1 (which produced notable faster code), and what about M1?

[Lasse Reinhold]

Sportman,

Can't you try to fix the bug for qpress 0.22? I'm eager to see if the new disk I/O (changed after 0.14) works as expected

[Sportman]

what about M1?

I don't know if this question is for me, but M1 had no problem at any test file.

[m^2]

Hey Sportman, I have some comments.

I love the idea, it's so awesome to be able to make own tests on a different machine, but some things could be improved.
1. It may be just a long queue, but possibly something hung again, I have some tests pending for a day and night. Could you look into it? And if such long queues happen, it would be nice if you showed how long (preferably in gigabytes) it is.
2. You have too many results to browse comfortably and things will be worse. Could you implement some filtering, i.e. to see only the latest archiver version or to remove failed modes?
I think that option to export to csv would be great too (probably even better).
3. I'd welcome a column separator.
4. How do you measure times? Some results look strange, i.e. file2, FreeArc -mdelta+rep:800mb+rzm has 6s decompression CPU time. rzm - 71.5s. rep can help, possibly delta too, but....

[m^2]

It's working now, but I have another request: Could you increase maximum command line length? I'm short of ~10 chars.

[Darek]

H!,
I've encountered some problems with adding paq8k version to metacompressor benchmarks.
I've tried to add paq8k compressor to "uiq2" benchmark, however after finishing submission steps and uploading archiver, I've got status message "Compression finished: 'uiq2' is compressed to 0 bytes in 0 seconds". I've checked out details and found that compressed file has 29 bytes - it's adequate value to paq8k header.
It looks like paq8k compressor run, but then, due to some reasons, it stopped and reports error.

What could be wrong? Command line looks proper. When I could find information about error reported by compressor?

Could you suggest me what should I do to run this benchmark Properly?

Regards, Darek