Page 4 of 4 FirstFirst ... 234
Results 91 to 111 of 111

Thread: enwik10 benchmark results

  1. #91
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    Quote Originally Posted by jibz View Post
    Could you add bcrush 0.2.0?
    Added.

  2. Thanks:

    jibz (7th February 2020)

  3. #92
    Member
    Join Date
    Mar 2013
    Location
    Worldwide
    Posts
    565
    Thanks
    67
    Thanked 199 Times in 147 Posts
    Quote Originally Posted by JamesB View Post
    It wasn't a critic, and I didn't believe the result shown was single threaded because you asked for it to be tested with multi-threading...
    It was a simple joke, I'm appreciating your comments and critics

    crashes on encode
    This is a test version and it's crashing for ex. with wrong parameters. Please use -t0 only.
    You can try -t6 for decoding. Multithreading is actually limited.
    The "-f" option is optional and it force deleting the output.
    Your original input file must not be in the current directory at decoding, it will be deleted otherwise.
    I've uploaded a TurboBWT linux version

  4. #93
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    Quote Originally Posted by Boris K View Post
    could you please test PPMd
    Added.

  5. Thanks:

    Boris K (6th March 2020)

  6. #94
    Member RichSelian's Avatar
    Join Date
    Aug 2011
    Location
    Shenzhen, China
    Posts
    171
    Thanks
    20
    Thanked 61 Times in 30 Posts
    could you also add orz v1.6.1, please? the source code is here: https://github.com/richox/orz/tree/v1.6.1

  7. #95
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    Quote Originally Posted by RichSelian View Post
    could you also add orz v1.6.1
    I need a Windows binary for that.

  8. #96
    Member RichSelian's Avatar
    Join Date
    Aug 2011
    Location
    Shenzhen, China
    Posts
    171
    Thanks
    20
    Thanked 61 Times in 30 Posts
    here it is
    Attached Files Attached Files

  9. Thanks (2):

    Mike (14th February 2020),Sportman (14th February 2020)

  10. #97
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    Quote Originally Posted by RichSelian View Post
    could you also add orz v1.6.1
    Added.

  11. #98
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    344
    Thanks
    50
    Thanked 62 Times in 50 Posts
    Quote Originally Posted by Sportman View Post
    Added.
    could you also add bbb v1.8 , please ?

  12. #99
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    Quote Originally Posted by suryakandau@yahoo.co.id View Post
    could you also add bbb v1.8
    Added default mode.

  13. Thanks:

    suryakandau@yahoo.co.id (16th February 2020)

  14. #100
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    344
    Thanks
    50
    Thanked 62 Times in 50 Posts
    Quote Originally Posted by Sportman View Post
    Added default mode.
    How about bbb cm1000 for v1.8 ?? i use bbb cm1000 for v1.8

  15. #101
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    Quote Originally Posted by suryakandau@yahoo.co.id View Post
    How about bbb cm1000 for v1.8 ?
    Added Shelwien compile.

  16. Thanks:

    suryakandau@yahoo.co.id (18th February 2020)

  17. #102
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    Quote Originally Posted by Sportman View Post
    I had planned to do a multi-core benchmark
    New 10 core test system can run 5.1GHz OC without noise, first multi-core benchmark results published.

  18. Thanks (2):

    Cyan (26th August 2020),encode (26th August 2020)

  19. #103
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    Quote Originally Posted by michael maniscalco View Post
    m99 d enwik10.m99 enwik10
    Elapsed time: 236.818 seconds
    Was your file compare successful?

    It fail (m99_gcc82_k8 version) with enwik10 also for 1 thread, enwik9 compare do not fail.

  20. #104
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    85
    Thanks
    15
    Thanked 5 Times in 5 Posts
    Quote Originally Posted by Sportman View Post
    New 10 core test system can run 5.1GHz OC without noise, first multi-core benchmark results published.
    How much the effect of test hardware on data compression ? How fast it is compared to old test hardware ?

  21. #105
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    Quote Originally Posted by bwt View Post
    How fast it is compared to old test hardware ?
    I have both systems for testing, old I use for single core enwik10 tests, new for multi core enwik10 tests (more memory, more cores, less noise).

    Old test system run OC 5.2GHz all cores (8 ) with 4266MHz memory (32GB) and 16GB cache, new test system run OC 5.1GHz all cores (10) (single core 5.3GHz) with 4000MHz memory (64GB) and 20MB cache, new test system is almost equal speed with this clock speed as old test system, sometimes a little faster, sometimes a little slower.

    New test system can handle OC 5.2GHz all cores or OC 5.5GHz some cores, then it's faster when used with these settings.
    Last edited by Sportman; 27th August 2020 at 21:04.

  22. #106
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    Quote Originally Posted by dnd View Post
    Can you please benchmark the new experimental with limited multithreading
    bwtturbo -59 -t0/bwtturbo -d -t0 with enwik10 file compare fail, bwtturbo -59 -t4/bwtturbo -d -t4 with enwik10 file compare do not fail.
    Last edited by Sportman; 27th August 2020 at 21:03.

  23. #107
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    85
    Thanks
    15
    Thanked 5 Times in 5 Posts
    Quote Originally Posted by Sportman View Post
    I have both systems for testing, old I use for single core enwik10 tests, new for multi core enwik10 tests (more memory, more cores, less noise).

    Old test system run OC 5.2GHz all cores (8 ) with 4266MHz memory (32GB) and 16GB cache, new test system run OC 5.1GHz all cores (10) (single core 5.3GHz) with 4000MHz memory (64GB) and 20MB cache, new test system is almost equal speed with this clock speed as old test system, sometimes a little faster, sometimes a little slower.

    New test system can handle OC 5.2GHz all cores or OC 5.5GHz some cores, then it's faster when used with these settings.

    i mean if i run a codec on old hardware the time is 2000 second, can it run faster on newer hardware eg down to 1000 second ?

  24. #108
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    Quote Originally Posted by bwt View Post
    i mean if i run a codec on old hardware the time is 2000 second, can it run faster on newer hardware eg down to 1000 second ?
    With this old and new hardware between 87%-718% speedup, depending single core or multi core or hyper threading is used.

    12 years old test system Intel Core2 Extreme Processor QX9650 OC 3.666GHz all cores (4) with 1333MHz memory (8GB) and 12MB cache:

    zstd -b1 -e22 enwik8 (v1.4.5):
    Code:
     1#enwik8            : 100000000 ->  40738526 (2.455), 147.5 MB/s , 587.9 MB/s
     2#enwik8            : 100000000 ->  37434671 (2.671), 135.5 MB/s , 501.8 MB/s
     3#enwik8            : 100000000 ->  35602698 (2.809), 110.0 MB/s , 481.5 MB/s
     4#enwik8            : 100000000 ->  34920953 (2.864), 101.4 MB/s , 470.5 MB/s
     5#enwik8            : 100000000 ->  34315441 (2.914),  55.7 MB/s , 446.2 MB/s
     6#enwik8            : 100000000 ->  33591155 (2.977),  44.6 MB/s , 460.6 MB/s
     7#enwik8            : 100000000 ->  32443027 (3.082),  32.5 MB/s , 492.1 MB/s
     8#enwik8            : 100000000 ->  32029317 (3.122),  26.1 MB/s , 513.0 MB/s
     9#enwik8            : 100000000 ->  31751130 (3.149),  18.5 MB/s , 520.2 MB/s
    10#enwik8            : 100000000 ->  31238385 (3.201),  11.7 MB/s , 511.4 MB/s
    11#enwik8            : 100000000 ->  30968326 (3.229),  8.41 MB/s , 509.1 MB/s
    12#enwik8            : 100000000 ->  30728213 (3.254),  5.77 MB/s , 516.0 MB/s
    13#enwik8            : 100000000 ->  30331609 (3.297),  5.60 MB/s , 526.2 MB/s
    14#enwik8            : 100000000 ->  29794982 (3.356),  4.15 MB/s , 518.2 MB/s
    15#enwik8            : 100000000 ->  29436073 (3.397),  3.17 MB/s , 502.4 MB/s
    16#enwik8            : 100000000 ->  28446554 (3.515),  3.13 MB/s , 521.1 MB/s
    17#enwik8            : 100000000 ->  27702787 (3.610),  2.25 MB/s , 477.9 MB/s
    18#enwik8            : 100000000 ->  27326331 (3.659),  1.89 MB/s , 431.7 MB/s
    19#enwik8            : 100000000 ->  26957405 (3.710),  1.68 MB/s , 388.2 MB/s
    20#enwik8            : 100000000 ->  25989368 (3.848),  1.39 MB/s , 453.8 MB/s
    21#enwik8            : 100000000 ->  25541097 (3.915),  1.24 MB/s , 455.5 MB/s
    22#enwik8            : 100000000 ->  25340452 (3.946),  1.15 MB/s , 455.9 MB/s

    New test system Intel i9-10900K OC 5.1GHz all cores (10) (single core 5.3GHz) with 4000MHz memory (64GB) and 20MB cache:

    zstd -b1 -e22 enwik8 (v1.4.5):
    Code:
     1#enwik8            : 100000000 ->  40738526 (2.455), 392.0 MB/s ,1351.2 MB/s
     2#enwik8            : 100000000 ->  37434671 (2.671), 263.3 MB/s ,1160.1 MB/s
     3#enwik8            : 100000000 ->  35602698 (2.809), 205.0 MB/s ,1072.2 MB/s
     4#enwik8            : 100000000 ->  34920953 (2.864), 198.7 MB/s ,1009.2 MB/s
     5#enwik8            : 100000000 ->  34315441 (2.914), 122.7 MB/s ,1002.3 MB/s
     6#enwik8            : 100000000 ->  33591155 (2.977),  89.0 MB/s ,1020.1 MB/s
     7#enwik8            : 100000000 ->  32443027 (3.082),  63.0 MB/s ,1086.1 MB/s
     8#enwik8            : 100000000 ->  32029317 (3.122),  48.6 MB/s ,1121.7 MB/s
     9#enwik8            : 100000000 ->  31751130 (3.149),  32.9 MB/s ,1122.7 MB/s
    10#enwik8            : 100000000 ->  31238385 (3.201),  27.9 MB/s ,1130.5 MB/s
    11#enwik8            : 100000000 ->  30968326 (3.229),  22.2 MB/s ,1135.4 MB/s
    12#enwik8            : 100000000 ->  30728213 (3.254),  13.7 MB/s ,1135.2 MB/s
    13#enwik8            : 100000000 ->  30331609 (3.297),  12.1 MB/s ,1156.8 MB/s
    14#enwik8            : 100000000 ->  29794982 (3.356),  9.07 MB/s ,1153.3 MB/s
    15#enwik8            : 100000000 ->  29436073 (3.397),  6.45 MB/s ,1185.7 MB/s
    16#enwik8            : 100000000 ->  28446554 (3.515),  6.57 MB/s ,1151.7 MB/s
    17#enwik8            : 100000000 ->  27702787 (3.610),  4.44 MB/s ,1151.7 MB/s
    18#enwik8            : 100000000 ->  27326331 (3.659),  3.73 MB/s ,1138.9 MB/s
    19#enwik8            : 100000000 ->  26957405 (3.710),  3.20 MB/s ,1103.3 MB/s
    20#enwik8            : 100000000 ->  25989368 (3.848),  2.59 MB/s ,1003.0 MB/s
    21#enwik8            : 100000000 ->  25541097 (3.915),  2.31 MB/s , 969.3 MB/s
    22#enwik8            : 100000000 ->  25340452 (3.946),  2.15 MB/s , 961.7 MB/s

    zstd and enwik8:

    Old system:
    Code:
    40,748,559 bytes,  0.234 sec. - 0.281 sec., zstd -1 -T4
    34,351,780 bytes,  0.920 sec. - 0.344 sec., zstd -5 -T4
    31,268,513 bytes,  4.695 sec. - 0.312 sec., zstd -10 -T4
    29,543,914 bytes, 14.071 sec. - 0.344 sec., zstd -15 -T4
    26,960,372 bytes, 28.127 sec. - 0.406 sec., zstd -19 -T4
    New system (hyper threading disabled):
    Code:
    40,748,559 bytes,  0.078 sec. - 0.109 sec., zstd -1 -T4
    34,351,780 bytes,  0.250 sec. - 0.156 sec., zstd -5 -T4
    31,268,513 bytes,  1.656 sec. - 0.140 sec., zstd -10 -T4
    29,543,914 bytes,  6.719 sec. - 0.125 sec., zstd -15 -T4
    26,960,372 bytes, 14.531 sec. - 0.140 sec., zstd -19 -T4
    
    40,748,559 bytes,  0.031 sec. - 0.109 sec., zstd -1 -T10
    34,351,780 bytes,  0.265 sec. - 0.141 sec., zstd -5 -T10
    31,268,513 bytes,  1.218 sec. - 0.125 sec., zstd -10 -T10
    29,543,914 bytes,  4.250 sec. - 0.125 sec., zstd -15 -T10
    26,960,372 bytes, 14.406 sec. - 0.141 sec., zstd -19 -T10
    New system (hyper threading enabled):
    Code:
    40,748,559 bytes,  0.047 sec. - 0.125 sec., zstd -1 -T20
    34,351,780 bytes,  0.250 sec. - 0.140 sec., zstd -5 -T20
    31,268,513 bytes,  1.250 sec. - 0.125 sec., zstd -10 -T20
    29,543,914 bytes,  4.281 sec. - 0.125 sec., zstd -15 -T20
    26,960,372 bytes, 14.640 sec. - 0.141 sec., zstd -19 -T20

    RAR and enwik8:

    Old system:
    Code:
    28,662,793 bytes, 31.559 sec. - 0.780 sec., rar -m3 -mt1
    
    28,662,797 bytes, 11.887 sec. - 0.608 sec., rar -m3 -mt4
    New system (hyper threading disabled):
    Code:
    28,662,793 bytes, 15.578 sec. - 0.406 sec., rar -m3 -mt1
    
    28,662,797 bytes,  4.891 sec. - 0.266 sec., rar -m3 -mt4
    
    28,663,089 bytes,  2.375 sec. - 0.234 sec., rar -m3 -mt10
    New system (hyper threading enabled):
    Code:
    28,663,077 bytes,  1.656 sec. - 0.234 sec., rar -m3 -mt20

    7z b:

    Old system:
    Code:
                           Compressing  |                  Decompressing
    Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
             KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS
    
    22:      15583   348   4360  15159  |     244383   398   5237  20849
    23:      14614   376   3965  14890  |     239873   398   5209  20756
    24:      13318   347   4122  14320  |     234226   400   5140  20555
    25:      crash
    New system (hyper threading disabled):
    Code:
                           Compressing  |                  Decompressing
    Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
             KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS
    
    22:      74233   890   8111  72214  |    1074602   992   9236  91645
    23:      69796   874   8132  71114  |    1054471   998   9143  91228
    24:      67290   906   7982  72350  |    1026779  1002   8992  90099
    25:      64161   893   8205  73257  |     995983   995   8903  88624
    ----------------------------------  | ------------------------------
    Avr:             891   8107  72234  |              997   9069  90399
    Tot:             944   8588  81316
    New system (hyper threading enabled):
    Code:
                           Compressing  |                  Decompressing
    Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
             KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS
    
    22:      92415  1652   5441  89902  |    1385735  1995   5921 118142
    23:      85653  1699   5137  87271  |    1367468  1997   5923 118297
    24:      83800  1802   5001  90102  |    1331773  1978   5907 116860
    25:      80551  1808   5086  91971  |    1318256  1991   5890 117290
    ----------------------------------  | ------------------------------
    Avr:            1740   5166  89812  |             1991   5911 117647
    Tot:            1865   5539 103729
    Last edited by Sportman; 30th August 2020 at 13:51.

  25. #109
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    Interesting to see that RAR beat zstd with compression speed because better core scaling:

    enwik8:
    29,543,914 bytes, 6.719 sec. - 0.125 sec., zstd -15 -T4
    29,543,914 bytes, 4.250 sec. - 0.125 sec., zstd -15 -T10
    29,543,914 bytes, 4.281 sec. - 0.125 sec., zstd -15 -T20 (hyper threading enabled)
    28,662,797 bytes, 4.891 sec. - 0.266 sec., rar -m3 -mt4
    28,662,880 bytes, 4.125 sec. - 0.250 sec., rar -m3 -mt5
    28,663,077 bytes, 1.656 sec. - 0.234 sec., rar -m3 -mt20 (hyper threading enabled)
    28,663,089 bytes, 2.375 sec. - 0.234 sec., rar -m3 -mt10
    28,472,688 bytes, 7.031 sec. - 0.125 sec., zstd -16 -T4
    28,472,688 bytes, 4.297 sec. - 0.125 sec., zstd -16 -T10

  26. #110
    Member
    Join Date
    Sep 2015
    Location
    Italy
    Posts
    278
    Thanks
    116
    Thanked 160 Times in 117 Posts
    Wrong order
    Code:
    2,872,160,117 bytes,    240.566 sec. -     20.319 sec., brotli -q 5 --large_window=30 (v1.0.7)
    2,915,934,603 bytes,    102.544 sec. -     13.302 sec., zstd -6 --ultra --single-thread (v1.4.4)
    2,915,934,603 bytes,    103.971 sec. -     12.798 sec., zstd -6 --ultra --single-thread (v1.4.5)
    2,812,779,013 bytes,    412.488 sec. -     64.311 sec., 7z -t7z -mx3 -mmt1 (v19.02)

  27. Thanks:

    Sportman (24th September 2020)

  28. #111
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    Quote Originally Posted by Mauro Vezzosi View Post
    Wrong order
    Fixed.

Page 4 of 4 FirstFirst ... 234

Similar Threads

  1. Benchmark results on GitHub
    By Piotr Tarsa in forum Data Compression
    Replies: 40
    Last Post: 30th May 2018, 05:47
  2. large window brotli results available for a several corpora
    By Jyrki Alakuijala in forum Data Compression
    Replies: 9
    Last Post: 3rd May 2018, 08:48
  3. Replies: 3
    Last Post: 30th July 2011, 14:48
  4. Strange gcc4.3 results with paq8o8
    By Hahobas in forum Forum Archive
    Replies: 8
    Last Post: 22nd March 2008, 19:44
  5. TC 5.1dev7x test results
    By LovePimple in forum Forum Archive
    Replies: 8
    Last Post: 23rd January 2007, 22:00

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •