Page 7 of 7 FirstFirst ... 567
Results 181 to 205 of 205

Thread: WCC - World Compressor Challenge - Incoming

  1. #181
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    902
    Thanks
    246
    Thanked 326 Times in 199 Posts
    Quote Originally Posted by load View Post
    large window brotli, windows GCC compile, if that helps.

    not tested.
    In practice this version only works with quality 11 and possibly with quality 10. Requires some patience to run on gigabytes of data... Decoding should be amazingly fast once the compression is ready.

  2. Thanks:

    Nania Francesco (24th October 2016)

  3. #182
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,572
    Thanks
    223
    Thanked 156 Times in 88 Posts
    large window brotli, windows GCC compile, if that helps.

    not tested.
    Attached Files Attached Files
    File Type: rar lw_brotli.rar (534.2 KB, 5 views)
    I tried the version that was compiled but not turned over this:

    --force --quality 10 --window 24 (option >=25 not work)

    Could you help me!

  4. #183
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,572
    Thanks
    223
    Thanked 156 Times in 88 Posts
    ZPAQ , MCM, LZHAM , FA 0.11 in testing ....

  5. #184
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,572
    Thanks
    223
    Thanked 156 Times in 88 Posts
    New release of WCC 2015-2016 is online
    added:
    - ZPAQ v.7.15
    - MCM v.0.83
    - LZHAM v.1.0
    - FA v.0.11
    only from:
    http://heartofcomp.altervista.org/MOC/MOCA.htm

  6. #185
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,572
    Thanks
    223
    Thanked 156 Times in 88 Posts
    Due to a problem in the log file where the logs of the benchmark that is corrupt, I take the trouble to anticipate the WCC's new version with some hope that 2017 corpus makes it more interesting!

  7. #186
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,572
    Thanks
    223
    Thanked 156 Times in 88 Posts
    I reduced benchmark to 1 GB for practical reasons (no time to test, I can test multiple options etc.). Among other things it is easily deduced the compression ratio.

  8. #187
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,572
    Thanks
    223
    Thanked 156 Times in 88 Posts
    I had to reduce the benchmark to 400 MB for the little time I have and to test multiple programs in multiple configurations. Soon released the first release of MOC 2017.

  9. #188
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,572
    Thanks
    223
    Thanked 156 Times in 88 Posts
    First release of WCC 2017 is online

    only from:
    http://heartofcomp.altervista.org/MOC/MOCA.htm

  10. Thanks (4):

    Christian (16th September 2017),Cyan (15th September 2017),Darek (16th September 2017),Kai Sandfort (15th September 2017)

  11. #189
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,572
    Thanks
    223
    Thanked 156 Times in 88 Posts
    First release of new WCC is online

    only from:
    http://heartofcomp.altervista.org/MOC/MOCA.htm

  12. Thanks (2):

    load (27th November 2018),Stephan Busch (1st December 2018)

  13. #190
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,572
    Thanks
    223
    Thanked 156 Times in 88 Posts
    I had to publish the news again as they deleted the latest posts! What happened Ilia?

  14. #191
    Member
    Join Date
    Apr 2009
    Location
    here
    Posts
    204
    Thanks
    172
    Thanked 110 Times in 66 Posts

  15. #192
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    902
    Thanks
    246
    Thanked 326 Times in 199 Posts
    Quote Originally Posted by Nania Francesco View Post
    I tried the version that was compiled but not turned over this:

    --force --quality 10 --window 24 (option >=25 not work)

    Could you help me!
    Try with --large_window 30

    The '--window' command line flag does not allow large window so that people don't accidentally compress their web resources with it. The web use aka "br" content encoding defined in RFC7932 limits window size to 16 MB.

  16. #193
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,572
    Thanks
    223
    Thanked 156 Times in 88 Posts
    First release of new WCC 2020-2021 is online.
    I'm coming back after a long break. I wasn't stopped. I have carried out some projects which I will soon publish! Meanwhile enjoy this new Benchmark that looks interesting and above all does not give so many advantages to text compressors !

    OS: Windows 10
    CPU: AMD Ryzen™ 3700x (8 core– 16 thread)

    only from:
    http://heartofcomp.altervista.org/MOC/MOCA.htm

  17. Thanks (5):

    algorithm (17th October 2020),comp1 (17th October 2020),fabiorug (17th October 2020),Lucas (17th October 2020),Mike (17th October 2020)

  18. #194
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,572
    Thanks
    223
    Thanked 156 Times in 88 Posts
    New update.

    Added
    - Thor 0.96a (Compressor)
    - Packet 1.9 (Archiver)
    - Razor 1.3.7 (Archiver)
    - ZCM 0.93 (Archiver)
    - ULZ 1.01B (Compressor)
    - Xeloz 0.3.5.3 (Compressor)


    only from:
    http://heartofcomp.altervista.org/MOC/MOCA.htm

  19. Thanks (2):

    fabiorug (17th October 2020),Mike (17th October 2020)

  20. #195
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    902
    Thanks
    246
    Thanked 326 Times in 199 Posts
    Quote Originally Posted by Nania Francesco View Post
    I tried the version that was compiled but not turned over this:

    --force --quality 10 --window 24 (option >=25 not work)

    Could you help me!
    Values above 24 are an extension to the RFC7932 and will not work together with browser's HTTP content encoding (because of decoding memory use limitations).

    This is why you need a special flag for large window.

    --large_window 30

    I believe only quality 11 (and possibly 10) max out the possibilities of the large window. Large window brotli would be ideal for distribution of packages, as it is highly asymmetric (super dense, but very fast to decode).

  21. #196
    Member
    Join Date
    Aug 2014
    Location
    Argentina
    Posts
    542
    Thanks
    239
    Thanked 93 Times in 73 Posts
    Good work Nania!

    Question: How do you decide which compressors to test? I see some heavy names missing, especially taking into consideration you include dead ones like nanozip, and slow configurations like -cc. What about paq8pxd(d) and precomp, for example?

    In the case of precomp, its worse-case scenario is a 7zip-like compression, and for a lot of other test sets it should take the first place, especially but not limited to pdf, mp3 and jpg.

    And if you include paq8px and paq8pxd, they should quickly gain #1 or #2 spots for basically each and every test (except maybe for mp3, which would still belong to precomp).

    You might also like to take a look at bcm, bsc for fast and strong symmetric compressors, and if you plan to keep nanozip, maybe you should include zpaq and freearc too. They are still competitive. As for closed source or propietary compressors, you might want to take a look at PowerArchiver and TC4Shell version of 7z, which includes a lot of great plugins for better compression, like wavpack for uncompressed audio.

  22. #197
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,572
    Thanks
    223
    Thanked 156 Times in 88 Posts
    Quote Originally Posted by Gonzalo View Post
    Good work Nania!

    Question: How do you decide which compressors to test? I see some heavy names missing, especially taking into consideration you include dead ones like nanozip, and slow configurations like -cc. What about paq8pxd(d) and precomp, for example?

    In the case of precomp, its worse-case scenario is a 7zip-like compression, and for a lot of other test sets it should take the first place, especially but not limited to pdf, mp3 and jpg.

    And if you include paq8px and paq8pxd, they should quickly gain #1 or #2 spots for basically each and every test (except maybe for mp3, which would still belong to precomp).

    You might also like to take a look at bcm, bsc for fast and strong symmetric compressors, and if you plan to keep nanozip, maybe you should include zpaq and freearc too. They are still competitive. As for closed source or propietary compressors, you might want to take a look at PowerArchiver and TC4Shell version of 7z, which includes a lot of great plugins for better compression, like wavpack for uncompressed audio.
    I will try to test the archivers and compressors as much as possible and accept your suggestions. As far as the "paq" series is concerned, I will do everything to test the program which, in my opinion, with the current processors is too slow and not very useful to be used in a practical way every day. Currently only Razor, Packet, Nanozip, 7zip and Rar, in my opinion, are very practical and useful archivers. The tests speak though!

  23. Thanks:

    Gonzalo (18th October 2020)

  24. #198
    Member
    Join Date
    Aug 2014
    Location
    Argentina
    Posts
    542
    Thanks
    239
    Thanked 93 Times in 73 Posts
    Fair enough! Thanks.

    Make sure you test the latest versions, though... For example, precomp has seen a dramatic increase in speed over the last few years. Nowadays it's usually faster than Razor, while providing better compression if there is something to recompress. And as far as practical compressors go, FreeArc is probably better than 7-zip even now after years of abandonment (faster & denser archives thanks to rep filter and tta among other things).

  25. #199
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    388
    Thanks
    57
    Thanked 77 Times in 58 Posts
    Quote Originally Posted by Nania Francesco View Post
    I will try to test the archivers and compressors as much as possible and accept your suggestions. As far as the "paq" series is concerned, I will do everything to test the program which, in my opinion, with the current processors is too slow and not very useful to be used in a practical way every day. Currently only Razor, Packet, Nanozip, 7zip and Rar, in my opinion, are very practical and useful archivers. The tests speak though!
    Maybe you could try Fp8sk since it is the fastest in paq series.

  26. #200
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,572
    Thanks
    223
    Thanked 156 Times in 88 Posts
    Due to an error encountered, I am testing the simple compressors again in "iso" format I am currently continuing with all the compressors already tested (ZSTD, LAZY etc).

    I'm sorry!

    I hope to republish soon !

  27. Thanks:

    danlock (23rd October 2020)

  28. #201
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    902
    Thanks
    246
    Thanked 326 Times in 199 Posts
    Quote Originally Posted by Nania Francesco View Post
    I will try to test the archivers and compressors as much as possible and accept your suggestions. As far as the "paq" series is concerned, I will do everything to test the program which, in my opinion, with the current processors is too slow and not very useful to be used in a practical way every day. Currently only Razor, Packet, Nanozip, 7zip and Rar, in my opinion, are very practical and useful archivers. The tests speak though!
    Sanmayce's testing from 2.5 years ago in https://github.com/google/brotli/issues/642 shows that large-window Brotli competes in density with 7zip, but decodes much faster.

    Brotli 11d29 is on pareto-optimal decoding speed/density for every large-corpora test, sometimes improving by more than 10x on the previous entry on pareto-front.

  29. #202
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,572
    Thanks
    223
    Thanked 156 Times in 88 Posts
    I was able to test all the archivers and compressors again and also added Lzpm 0.18 and Precomp 0.4.7. I hope to release more updates soon!


    only from:
    http://heartofcomp.altervista.org/MOC/MOCA.htm

  30. Thanks:

    schnaader (Yesterday)

  31. #203
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    616
    Thanks
    263
    Thanked 242 Times in 121 Posts
    Thanks for adding Precomp to the benchmark! Could you change the country from RUS to GER?

    Some suggestions for options that might improve results for some of the tests:
    "-cl -intense" for summary, app1, app2, game1, game2, iso, raw, bin tests (intense is a bit slower, but detects headerless zlib streams)
    "-cl -lf+d1", "-cl -lf+d2" or "-cl -lf+d4" for wav test (enables lzma2 delta compression which will often lead to better results on audio)
    http://schnaader.info
    Damn kids. They're all alike.

  32. #204
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,572
    Thanks
    223
    Thanked 156 Times in 88 Posts
    Quote Originally Posted by schnaader View Post
    Thanks for adding Precomp to the benchmark! Could you change the country from RUS to GER?

    Some suggestions for options that might improve results for some of the tests:
    "-cl -intense" for summary, app1, app2, game1, game2, iso, raw, bin tests (intense is a bit slower, but detects headerless zlib streams)
    "-cl -lf+d1", "-cl -lf+d2" or "-cl -lf+d4" for wav test (enables lzma2 delta compression which will often lead to better results on audio)
    As soon as I can get my hands on the resource files I use for the construction of html pages ! I'm sorry I got the nation wrong!

  33. #205
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,572
    Thanks
    223
    Thanked 156 Times in 88 Posts
    Quote Originally Posted by schnaader View Post
    Thanks for adding Precomp to the benchmark! Could you change the country from RUS to GER?

    Some suggestions for options that might improve results for some of the tests:
    "-cl -intense" for summary, app1, app2, game1, game2, iso, raw, bin tests (intense is a bit slower, but detects headerless zlib streams)
    "-cl -lf+d1", "-cl -lf+d2" or "-cl -lf+d4" for wav test (enables lzma2 delta compression which will often lead to better results on audio)
    I have managed to change yours nationality! Sorry I'm late!

Page 7 of 7 FirstFirst ... 567

Similar Threads

  1. a challenge for C experts
    By Alexander Rhatushnyak in forum The Off-Topic Lounge
    Replies: 10
    Last Post: 25th January 2012, 04:29
  2. Calgary challenge
    By Matt Mahoney in forum Data Compression
    Replies: 6
    Last Post: 12th September 2010, 01:45
  3. The lie of "The world is a globe"
    By Vacon in forum The Off-Topic Lounge
    Replies: 2
    Last Post: 14th December 2009, 15:58
  4. WinTURTLE 1.0.0 Beta incoming!
    By Nania Francesco in forum Forum Archive
    Replies: 70
    Last Post: 27th January 2008, 00:57
  5. TURTLE incoming... Fast PPM file compressor.
    By Nania Francesco in forum Forum Archive
    Replies: 104
    Last Post: 8th August 2007, 20:40

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •