Results 1 to 18 of 18

Thread: Metacompressor.com first contest

  1. #1
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    782
    Thanks
    63
    Thanked 273 Times in 191 Posts

    Metacompressor.com first contest

    I just added a new test file “file11” size 347MB and I do not give away the data type. This test file is only visible after login via the “add archiver” link.

    Login:
    http://www.metacompressor.com/login.aspx

    Everybody can try to compress this test file “file11” as small as possible there are 6 categories:

    Compression time:
    < 300 seconds
    < 120 seconds
    < 50 seconds
    < 18 seconds
    < 10 seconds
    < 3 seconds

    Decompression and compare must be successful.

    You can use any archiver out there or create one yourself.

    After login and selecting “file11” you can see your own results. It’s not possible to see the results of others, they shall be visible after the contest has finished

    The contest ends October 12, 2008 at 24:00:00 GMT.

    I’m excluded from this contest as competitor but shall be jury and make results lists after the contest has finished.

    There is no price money or goods.

  2. #2
    Member
    Join Date
    May 2008
    Location
    Antwerp , country:Belgium , W.Europe
    Posts
    487
    Thanks
    1
    Thanked 3 Times in 3 Posts
    It seems that new uploaded archivers do not appear in the "personal" list after logging in....
    To test, I uploaded BZP v0.3 and created a simple archive command with it.
    After logging in and checking "file11", it wasn't there.
    Could you please have a look at this ? Thanks !

  3. #3
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    782
    Thanks
    63
    Thanked 273 Times in 191 Posts
    Quote Originally Posted by pat357 View Post
    It seems that new uploaded archivers do not appear in the "personal" list after logging in....
    There is no "personal" archiver list yet, it must be uploaded manual every test.

    New archivers uploaded as "public" shall only appear in the "public" archiver list when the test is started, as long it's queued it's not visible.

    Only "personal" results of contest "file11" tests are visible after login at that same page after selecting “file11“, there is no other place where "file11" results can be seen.

    Only the "add archiver" link after login lead to a submit archiver page with "file11" listed under test files. This same link automatic fill in the code and email field so they are not displayed what save typing.

    Update1: I think I understand what you mean, the queued new archiver is not listed, I go to check this now.
    Update2: I reproduced this bug.
    Update3: fixed.
    Last edited by Sportman; 12th October 2008 at 20:52.

  4. #4
    Member
    Join Date
    May 2008
    Location
    Antwerp , country:Belgium , W.Europe
    Posts
    487
    Thanks
    1
    Thanked 3 Times in 3 Posts
    Quote Originally Posted by Sportman View Post
    Update1: I think I understand what you mean, the queued new archiver is not listed, I go to check this now.
    Update2: I reproduced this bug.
    Update3: fixed.
    Thanks a lot !
    Please check your PM !

  5. #5
    Programmer
    Join Date
    May 2008
    Location
    denmark
    Posts
    94
    Thanks
    0
    Thanked 2 Times in 2 Posts

    Hangs at "Busy unzipping archiver"

    When trying to upload qpress020.zip it keeps saying "Busy unzipping archiver". It never changes status.

  6. #6
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    782
    Thanks
    63
    Thanked 273 Times in 191 Posts
    Quote Originally Posted by Lasse Reinhold View Post
    When trying to upload qpress020.zip it keeps saying "Busy unzipping archiver". It never changes status.
    When there is no archiver name in the command line compress or decompress it stop trying to compress. I have added some extra command line input checks and added a status update to “compression failed, can be a parameter mistake” to make it more clear.

  7. #7
    Programmer
    Join Date
    May 2008
    Location
    denmark
    Posts
    94
    Thanks
    0
    Thanked 2 Times in 2 Posts

    Hangs again

    Now it hangs again, this time at "Busy compressing file1. "

    What did I do wrong this time?

  8. #8
    Programmer osmanturan's Avatar
    Join Date
    May 2008
    Location
    Mersin, Turkiye
    Posts
    651
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Question

    What about the competition result?

  9. #9
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    782
    Thanks
    63
    Thanked 273 Times in 191 Posts
    Quote Originally Posted by Lasse Reinhold View Post
    Now it hangs again, this time at "Busy compressing file1. "

    What did I do wrong this time?
    Nothing, compression finished and status updated but decompression failed and updated status was not showed right in the status list, I fixed this.

    The compressed output file was only 47 bytes and that's why decompression failed. Again here is a strange problem because when I run the same compress command line under the command prompt it works fine and the output has a normal size. Some archivers fail to run under the windows test control application and I don't know why and how to solute it

  10. #10
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    782
    Thanks
    63
    Thanked 273 Times in 191 Posts

    Thumbs up

    Quote Originally Posted by osmanturan View Post
    What about the competition result?
    I just see I made a mistake with the end data to make it fair I stopped it now and made the result list, there is one very clear winner Pat357 congratulations you won all the categories place 1 till 3!

    <3 seconds:
    1) Pat357 - 87,200,916 bytes 2.583 sec. slug 1.27b - slug c @infile @outfile
    2) Pat357 - 94,938,220 bytes 2.975 sec. thor 0.96 alpha - thor e3 @infile @outfile
    3) Pat357 - 97,286,840 bytes 2.275 sec. thor 0.96 alpha - thor e2 @infile @outfile

    <10 seconds:
    1) Pat357 - 72,376,865 bytes 6.645 sec. bzp 0.3 - bzp c @infile @outfile
    2) Pat357 - 87,200,916 bytes 2.583 sec. slug 1.27b - slug c @infile @outfile
    3) Pat357 - 89,181,676 bytes 6.713 sec. thor 0.96 alpha - thor e4 @infile @outfile

    <18 seconds:
    1) Pat357 - 60,217,919 bytes 16.543 sec. 4x4 v0.2a - 4x4 grzip:m1:h20 @infile @outfile
    2) Pat357 - 67,775,810 bytes 13.695 sec. sr3a 3a - sr3a c @infile @outfile
    3) Pat357 - 70,994,702 bytes 16.796 sec. 7za 4.60 7za a -tbzip2 -mx5 -mmt=4 -o@outdir @outfile @infile

    <50 seconds:
    1) Pat357 - 52,293,656 bytes 46.111 sec. 7za 4.60 - 7za a -t7z -m0=ppmd:o=10:mem=1650m -o@outdir @outfile @infile
    2) Pat357 - 52,319,420 bytes 43.661 sec. arc 0.5a - arc a -mppmd:10:1700m -lc1700m -ld1700m @outfile @infile
    3) Pat357 - 52,396,379 bytes 42.45 sec. arc 0.5a - arc a -mppmd:9:1700m -lc1700m -ld1700m @outfile @infile

    <120 and <300 seconds:
    1) Pat357 - 51,081,296 bytes 221 sec. FreeArc v0.50 - arc a -mpmm:10:1700m -lc1700mb -ld1700mb @outfile @infile
    2) Pat357 - 51,316,459 bytes 161 sec. FreeArc v0.50 - arc a -mpmmi:10 -lc1650mb -ld1650mb @outfile @infile
    3) Pat357 - 52,293,656 bytes 46.111 sec. 7za 4.60 - 7za a -t7z -m0=ppmd:o=10:mem=1650m -o@outdir @outfile @infile

  11. #11
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    735
    Thanked 660 Times in 354 Posts
    1. are you tried to use 4x4 3/4/5/6 modes? i think it should be a good competition for thor and slug. and "4x4 grzip:m4:h20" may be a winner in "<10" category

    2. are you tried to add dict and lzp preprocessors to ppmd/pmm?

    3. what's pmmi?
    Last edited by Bulat Ziganshin; 14th October 2008 at 13:36.

  12. #12
    Member
    Join Date
    May 2008
    Location
    Antwerp , country:Belgium , W.Europe
    Posts
    487
    Thanks
    1
    Thanked 3 Times in 3 Posts
    Quote Originally Posted by Bulat Ziganshin View Post
    1. are you tried to use 4x4 3/4/5/6 modes? i think it should be a good competition for thor and slug. and "4x4 grzip:m4:h20" may be a winner in "<10" category
    Haven't tried yet, but l do it.
    2. are you tried to add dict and lzp preprocessors to ppmd/pmm?
    Indeed, I've tried both, they gave worse compression.
    This had puzzled me as on what kind of data (txt ?) dict / lzp preprocessing would give worse ratio's.
    On plain txt dict/lzp should almost always improve both ratio and speed, so my guess is that this file11 is not "plain txt".
    This is probably also the reason why Nanozip -co/cO is not in the "top-list". I guess that the preprocessing from Nanozip was really hurting the comp.ratio for this specific file.
    On "plain txt" NZ -co/-cO would probably blow everything away, both on speed and ratio.
    Also, ARC with "auto-detection" seems to use LZMA to compress this file : arc -m7 -> comp=249s / decomp. = 10s.
    3. what's pmmi?
    PPMonstr vI . It's seems to be often faster than PPMonster J and compression with PPMonstrI is only a tad less than PPMonstrJ.
    Last edited by pat357; 14th October 2008 at 15:54.

  13. #13
    Member
    Join Date
    May 2008
    Location
    Antwerp , country:Belgium , W.Europe
    Posts
    487
    Thanks
    1
    Thanked 3 Times in 3 Posts
    Quote Originally Posted by Sportman View Post
    I just see I made a mistake with the end data to make it fair I stopped it now and made the result list, there is one very clear winner Pat357 congratulations you won all the categories place 1 till 3!
    Thanks !
    What kind of data is file11 ?
    It behaves like txt-like data, but based on the results it seems not to be English plain-txt. (see also my reply to Bulat).

  14. #14
    Programmer
    Join Date
    May 2008
    Location
    denmark
    Posts
    94
    Thanks
    0
    Thanked 2 Times in 2 Posts
    Hi Sportman,

    Could it be because the "current/active/whatever it's called" dir of your script is different from that of the compressor so that the compressor can't see the input files?

    It's a common problem when executing other programs from scripts, and current/active dir can some times change arbitrarely at random, depending on how and from where you start the script and other things.

    Try giving full path names to the compressor.
    Last edited by Lasse Reinhold; 14th October 2008 at 16:24.

  15. #15
    Member
    Join Date
    May 2008
    Location
    Antwerp , country:Belgium , W.Europe
    Posts
    487
    Thanks
    1
    Thanked 3 Times in 3 Posts
    Quote Originally Posted by Sportman View Post
    Again here is a strange problem because when I run the same compress command line under the command prompt it works fine and the output has a normal size. Some archivers fail to run under the windows test control application and I don't know why and how to solute it
    Could this also be the reason that CCM/CCMx fail to compress some files (like your file 11) ?
    I noticed that for CCM(x), it seems to work for smaller files,but not for the larger ones.... very strange..

    edit :
    another one that fails with a strange error :
    Code:
    quicklz	1.4.0 beta 9	quick2 @infile @outfile	quick2 -d @infile @outdir	10/13/2008 3:03:18 PM	Test system error 3
    Any clue about this Test system error 3 ?
    Last edited by pat357; 14th October 2008 at 19:06.

  16. #16
    Member
    Join Date
    May 2008
    Location
    Antwerp , country:Belgium , W.Europe
    Posts
    487
    Thanks
    1
    Thanked 3 Times in 3 Posts
    Quote Originally Posted by pat357 View Post
    Haven't tried yet, but l do it.
    Done.
    Code:
    60,217,919	16.543	00:01:03.664008     4x4 grzip:m1:h20
    66,248,461	6.493	00:00:17.4409118    4x4 grzip:m4:h20
    83,906,577	4.983	00:00:17.0041090    4x4 6 
    85,966,869	3.221	00:00:11.1852717    4x4 5 
    91,582,008	1.947	00:00:05.0856326    4x4 4 
    96,084,036	1.654	00:00:03.0420195    4x4 3
                                        
    87,200,916	2.583	00:00:02.5740165    slug c 
    89,181,676      6.713                       thor e4
    94,938,220      2.975                       thor e3 
    97,286,840      2.275                       thor e2
    72,376,865      6.645       bzp 0.3         bzp c
    4x4 grzip:m4:h20 indeed takes the lead for <10s.

  17. #17
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    782
    Thanks
    63
    Thanked 273 Times in 191 Posts
    Quote Originally Posted by pat357 View Post
    Thanks !
    What kind of data is file11 ?
    It behaves like txt-like data, but based on the results it seems not to be English plain-txt. (see also my reply to Bulat).
    You are right it's a text file with .txt file extension.

    File11 is a list of all tld .NET unique domains as URL there exist, almost 12 million plus round 300k with sub domain.

    A friend noticed this type of data was difficult to compress.

    I expected it could be compressed round 27MB but he was right and 52MB was the max I reached with existing archivers.

  18. #18
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    735
    Thanked 660 Times in 354 Posts
    i thought it's something like dictionary - all these features are typical for dicts. i was not entirely incorrect :)

    you can improve grzip results by adding ":l" to settings

    while dict doesn't help, compression of such data may be better with wrt-type preprocessor

    ppmonstr h-i-j each version improved compression by 5% and decreased speed 2x. i personally prefer I

Similar Threads

  1. Matlab Contest: Compressed Sensing
    By russelms in forum Data Compression
    Replies: 1
    Last Post: 28th April 2010, 22:48
  2. the latest rec math contest is just starting
    By willvarfar in forum The Off-Topic Lounge
    Replies: 0
    Last Post: 15th March 2010, 00:12
  3. Metacompressor.com benchmark updates
    By Sportman in forum Data Compression
    Replies: 79
    Last Post: 22nd April 2009, 03:24

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •