Results 1 to 13 of 13

Thread: Oodle Algorithm Benchmarks

  1. #1
    Member
    Join Date
    May 2012
    Location
    United States
    Posts
    331
    Thanks
    190
    Thanked 55 Times in 39 Posts

    Oodle Algorithm Benchmarks

    I've done some benchmarks of the various algorithms included with Oodle.

    Tests were done on my Win7 x64, 32 GB RAM, i7-3770k machine. Times were taken using Igor's timer32 (Version 14.00).

    * Syntax used:
    LZMA : -mx=9 -m0=lzma
    ZSTD : -22 --ultra

    ENWIK9
    Code:
      ALGORITHM  SIZE       C.TIME    D.TIME
    ==========================================
            LZA  204849620  3491.614  19.827
           LZNA  207399996  3141.111  15.615
         Kraken  212731709  2944.690  10.249
          LZHLW  222405577  4478.695  11.403
          LZMA*  213336029   735.919  10.483
          ZSTD*  215629659   790.987   3.978
        BitKnit  229587592   370.455  11.700
        Mermaid  264586873  2679.426   9.859
          LZNIB  292688201   209.743  15.600
         Selkie  292601419  2681.469  16.988
            LZH  303288757   103.756   9.952
          LZBLW  310493874  4620.827   9.765
          LZB16  372018973   239.789   8.720
    SFC
    Code:
      ALGORITHM  SIZE      C.TIME    D.TIME
    ==========================================
          LZMA*  12402317    12.854    0.608
            LZA  12502228    42.354    0.842
          ZSTD*  12909223    19.297    0.109
          LZHLW  12952200    71.479    0.312
           LZNA  13018920    28.891    0.436
        BitKnit  13035440    21.262    0.358
            LZH  14278381     2.870    0.296
        Mermaid  14742844    15.085    0.312
          LZNIB  15249478     5.335    0.265
         Selkie  16299931    15.303    0.202
          LZBLW  16487244    60.403    0.280
          LZB16  16842038    10.623    0.265
         Kraken  - crash -
    vm.dll (HFCB)
    Code:
      ALGORITHM  SIZE        C.TIME    D.TIME
    ===========================================
          LZMA*   961424977  1195.030   52.073
            LZA   963634787  7440.108  105.690
           LZNA   972415221  6610.682   63.929
         Kraken  1006969847  6315.466   32.931
          LZHLW  1037603623  8837.846   41.246
          ZSTD*  1042158230  1431.496    8.658
        BitKnit  1047215335   736.558   38.532
        Mermaid  1197207335  5820.724   31.184
          LZNIB  1207160786   439.252    3.556
            LZH  1306548185   209.446   36.473
         Selkie  1313162284  5761.522   31.949
          LZBLW  1331752046  9257.473    2.948
          LZB16  1571255600   897.816    2.620
    All tests were done with compression level 9.
    Last edited by comp1; 2nd August 2016 at 14:38.

  2. Thanks (3):

    algorithm (2nd August 2016),encode (24th August 2018),Minimum (31st July 2016)

  3. #2
    Member
    Join Date
    Dec 2015
    Location
    US
    Posts
    57
    Thanks
    2
    Thanked 112 Times in 36 Posts
    Look, Oodle is commercial software. If you really feel the need to grab the DLL from somewhere else and run your own benchmarks, then please at least refrain from putting said DLL along with a reverse-engineered stub header file on a public server and posting a link on a public forum, okay?

    That's an Oodle 2.2.0 DLL. It contains algorithms named "Mermaid" and "Selkie", but in 2.2.0 they're undocumented, don't show up in the official header file, and they are not what we recently released as Oodle Mermaid and Oodle Selkie in 2.3.0. They're earlier prototypes from several bitstream revisions ago, not fuzz safe, and their output cannot be decoded by Oodle 2.3.0 (or any other upcoming releases). The other codecs were released with a stable bitstream at the time.

    Oodle isn't zlib; compresion level 9 in Oodle isn't even a thing - you can specify it, but it just gets clamped to a lower number. No codec implements anything higher than 8, which is already well into "completely impractical" territory. All of the really high levels are primarily for internal use in development, as a baseline. The faster-to-encode codecs in your list are simply the ones that don't implement the high levels to begin with and snap to something lower. Useful levels to try are 2 (fastest) through 6 (high compression). 0 and 1 are special, and 7/8 are implemented in some codecs if someone wants them but generally too slow to be interesting. 4-6 are the workhorse levels. The numbers we post are generally levels 5 or 6.

  4. Thanks (2):

    Malloc Voidstar (2nd August 2016),Minimum (31st July 2016)

  5. #3
    Member
    Join Date
    May 2012
    Location
    United States
    Posts
    331
    Thanks
    190
    Thanked 55 Times in 39 Posts
    @fgiesen:

    My apologies. This was not intended to be anything more than a simple benchmark thread to soothe curious minds. The link has been removed from my original post.

  6. #4
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,760
    Thanks
    274
    Thanked 1,200 Times in 668 Posts
    @comp1: Can you add lzma for comparison?
    Not one with filters though - either lzma.exe or 7z a -mx=9 -m0=lzma
    Also maybe zstd?

  7. #5
    Member
    Join Date
    May 2012
    Location
    United States
    Posts
    331
    Thanks
    190
    Thanked 55 Times in 39 Posts
    Quote Originally Posted by Shelwien View Post
    @comp1: Can you add lzma for comparison?
    Not one with filters though - either lzma.exe or 7z a -mx=9 -m0=lzma
    Also maybe zstd?
    Good idea!

    LZMA and ZSTD added.

  8. #6
    Member
    Join Date
    Nov 2013
    Location
    Kraków, Poland
    Posts
    747
    Thanks
    232
    Thanked 235 Times in 145 Posts
    Fabian has written that level 9 "isn't even a thing", that usable are levels 4-6, maybe you could change to level 6?

  9. #7
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,760
    Thanks
    274
    Thanked 1,200 Times in 668 Posts
    "level 9" is still usable and decodable, its just the same as level 8.
    Also level doesn't affect decoding speed, I think.

    Also I'd say level 4 is not a good idea, because there's a significant drop in compression between 4 and 5, and then 5 and 6 too.
    Well,
    Code:
    LZNA/book1
    level:      1       2       3       4       5       6       7       8       9
    csize: 406856. 363057. 319667. 314196. 281561. 267963. 267897. 267897. 267897. 
    ctime:  0.047   0.047   0.078   0.093   0.421   0.484   0.842   0.827   0.827 
    dtime:  0.015   0.000   0.016   0.000   0.016   0.000   0.000   0.015   0.016

  10. #8
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,537
    Thanks
    758
    Thanked 676 Times in 366 Posts
    i think that without using the same dictsize the comparison doesn't make much sense. for 7-zip it may be set by -md64m f.e. anso use -mmt2 to ensure that 7-zip will not split data into independent chunks

  11. #9
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,760
    Thanks
    274
    Thanked 1,200 Times in 668 Posts
    It doesn't split data into chunks for lzma, and d=64M is set by -mx=9, though I already suggested to use d=1G instead, because oodle there has dictionary=filesize for most codecs.

  12. #10
    Member
    Join Date
    Apr 2015
    Location
    Greece
    Posts
    84
    Thanks
    34
    Thanked 26 Times in 17 Posts
    Why is zstd so fast compared to the other compressors?Bad I/O or oodle is just a marketing hype?

  13. #11
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,760
    Thanks
    274
    Thanked 1,200 Times in 668 Posts
    bad i/o mostly. oodle only provides memory-to-memory compression (or I didn't find how to use it with file i/o, anyway),
    so this frontend reads whole file to memory, (de)compresses it in memory, then writes whole file.
    Also this oodle 2.20 supposedly has some MT, but I didn't test it yet, and this benchmark doesn't use it.

    Also zstd is just that good, I suppose? :)

  14. #12
    Member
    Join Date
    Dec 2011
    Location
    Cambridge, UK
    Posts
    487
    Thanks
    172
    Thanked 168 Times in 115 Posts
    It was pointed out that the newer Oodle algorithms weren't finished (or indeed officially available) in the version used in the tests above. Ideally we'd see benchmarks of the newer code instead, as this whole thread is probably a bit misleading; or at least grey out the selkie and mermaid lines.

  15. #13
    Member
    Join Date
    Nov 2015
    Location
    ?l?nsk, PL
    Posts
    81
    Thanks
    9
    Thanked 13 Times in 11 Posts
    Quote Originally Posted by JamesB View Post
    It was pointed out that the newer Oodle algorithms weren't finished (or indeed officially available) in the version used in the tests above. Ideally we'd see benchmarks of the newer code instead, as this whole thread is probably a bit misleading; or at least grey out the selkie and mermaid lines.
    Or better - write explicitly that these are prototypes.

Similar Threads

  1. zpaq benchmarks
    By Sportman in forum Data Compression
    Replies: 100
    Last Post: 17th December 2018, 22:48
  2. new compressor LZNA = "LZ-nibbled-ANS" - "Oodle 1.45"
    By joerg in forum Data Compression
    Replies: 22
    Last Post: 19th February 2018, 04:50
  3. Why more people need to do benchmarks
    By calthax in forum Data Compression
    Replies: 2
    Last Post: 17th August 2014, 23:37
  4. Greetings, Questions, and Benchmarks
    By musicdemon in forum Data Compression
    Replies: 4
    Last Post: 8th January 2012, 21:45
  5. MaximumCompression.com Benchmarks
    By osmanturan in forum Data Compression
    Replies: 29
    Last Post: 5th May 2009, 10:31

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •