Results 1 to 3 of 3

Thread: Multithread benchmarking

  1. #1
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts

    Multithread benchmarking

    Multithread is a program to benchmark single thread archivers multi thread.

    Download (.NET 4.61 version 64-bit/32-bit):
    http://www.metacompressor.com/download/multithread.zip

    Download (.NET 3.5 version 32-bit):
    http://www.metacompressor.com/downlo...read-32bit.zip

    For single thread testing add archiver(s) and test file(s) in same folder or root.
    For multi thread testing add archiver(s) and test file(s) in root for every thread one drive.

    Syntax examples:

    multithread archiver level threads file

    multithread brotli 1 enwik8 (brotli, level 1, single thread test)
    multithread brotli 1 1 enwik8 (brotli, level 1, single thread test)
    multithread brotli 1 2 enwik8 (brotli, level 1, 2 threads test, test in root drive started and next drive letter)
    multithread brotli 1 2 -dE enwik8 (brotli, level 1, 2 threads test, test in root drive E and F)
    multithread brotli 1 2 -dE -h enwik8 (brotli, level 1, 2 threads test, test in root drive E and F, hide thread command line output)
    multithread zstandard 1 4 -dE -h enwik8 (zstandard, level 1, 4 threads test, test in root drive E, F, G and H, hide thread command line output)

    Archivers supported:
    rar, 7zip (7z, zip, bzip2, xz), freearc, qpress, zstd, brotli, lzturbo

  2. Thanks:

    Jyrki Alakuijala (20th September 2016)

  3. #2
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    Updated Multithread, added lz4, lz5, zhuff and lzham support.

  4. #3
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    To create a benchmark output table as here:
    http://encode.su/threads/2577-Open-s...pression/page4

    Download example (single thread) batch file:
    http://www.metacompressor.com/download/mtt.zip

    Start mtt.bat batch file with multithread tests:
    mtt.bat > mtt.txt

    Download mtt.bat batch file mtt.txt output parser mtparser:

    Download (.NET 4.61 version 64-bit/32-bit):
    http://www.metacompressor.com/download/mtparser.zip

    Download (.NET 3.5 version 32-bit):
    http://www.metacompressor.com/downlo...rser-32bit.zip

    Parse mtt.txt output:
    mtparser mtt.txt mttr.txt

    Example of result in mttr.txt:
    rar 1 1 enwik8 40565271 bytes 2.003 0.703 2.007 0.703 2.008 0.703
    freearc 1 1 enwik8 38003873 bytes 1.164 0.656 1.164 0.656 1.164 0.656
    lzturbo 10 1 enwik8 52740824 bytes 0.382 0.093 0.382 0.093 0.382 0.093
    zstandard 1 1 enwik8 40859486 bytes 0.554 0.187 0.554 0.187 0.554 0.187
    brotli 1 1 enwik8 42456665 bytes 0.730 0.354 0.730 0.354 0.731 0.354
    qpress 1 1 enwik8 54294428 bytes 0.398 0.296 0.398 0.296 0.398 0.296
    7zip 1 1 enwik8 34707520 bytes 5.164 1.453 5.164 1.453 5.164 1.453
    zip 1 1 enwik8 37907740 bytes 2.507 0.624 2.508 0.624 2.508 0.624
    bzip2 1 1 enwik8 33253096 bytes 9.195 1.499 9.226 1.499 9.211 1.499
    xz 1 1 enwik8 34707456 bytes 5.226 1.453 5.179 1.453 5.179 1.453
    lz4 1 1 enwik8 53510696 bytes 0.101 0.078 0.101 0.078 0.101 0.078
    lz5 1 1 enwik8 71753612 bytes 0.273 0.140 0.273 0.140 0.273 0.140
    zhuff 0 1 enwik8 37053030 bytes 0.570 0.161 0.565 0.160 0.565 0.160
    lzham 0 1 enwik8 37981857 bytes 6.612 0.687 6.679 0.687 6.680 0.687

    Same batch file but threads changed in 2:
    rar 1 2 enwik8 40565271 bytes 2.044 0.718 2.039 0.718 2.054 0.718
    freearc 1 2 enwik8 38003873 bytes 1.179 0.656 1.194 0.656 1.178 0.671
    lzturbo 10 2 enwik8 52740824 bytes 0.491 0.109 0.398 0.109 0.413 0.109
    zstandard 1 2 enwik8 40859486 bytes 0.554 0.187 0.554 0.203 0.553 0.187
    brotli 1 2 enwik8 42456665 bytes 0.726 0.374 0.726 0.374 0.726 0.374
    qpress 1 2 enwik8 54294428 bytes 0.398 0.312 0.398 0.312 0.413 0.317
    7zip 1 2 enwik8 34707520 bytes 5.174 1.453 5.179 1.468 5.179 1.468
    zip 1 2 enwik8 37907740 bytes 2.523 0.625 2.522 0.625 2.523 0.624
    bzip2 1 2 enwik8 33253096 bytes 9.226 1.500 9.210 1.499 9.226 1.499
    xz 1 2 enwik8 34707456 bytes 5.226 1.468 5.179 1.468 5.289 1.468
    lz4 1 2 enwik8 53510696 bytes 0.132 0.093 0.133 0.093 0.132 0.093
    lz5 1 2 enwik8 71753612 bytes 0.289 0.156 0.288 0.156 0.289 0.156
    zhuff 0 2 enwik8 37053030 bytes 0.679 0.171 0.570 0.171 0.569 0.171
    lzham 0 2 enwik8 37982558 bytes 7.007 0.687 6.944 0.687 6.913 0.687

    Same batch file but threads changed in 4:
    rar 1 4 enwik8 40565271 bytes 2.191 0.734 2.132 0.734 2.147 0.734
    freearc 1 4 enwik8 38003873 bytes 1.319 0.718 1.288 0.718 1.287 0.749
    lzturbo 10 4 enwik8 52740824 bytes 0.413 0.125 0.472 0.125 0.444 0.125
    zstandard 1 4 enwik8 40859486 bytes 0.600 0.203 0.554 0.203 0.582 0.203
    brotli 1 4 enwik8 42456665 bytes 0.757 0.375 0.741 0.390 0.741 0.406
    qpress 1 4 enwik8 54294428 bytes 0.413 0.312 0.413 0.327 0.413 0.312
    7zip 1 4 enwik8 34707520 bytes 5.224 1.500 5.210 1.468 5.210 1.484
    zip 1 4 enwik8 37907740 bytes 2.523 0.640 2.538 0.640 2.600 0.640
    bzip2 1 4 enwik8 33253096 bytes 9.292 1.527 9.272 1.515 9.241 1.515
    xz 1 4 enwik8 34707456 bytes 5.210 1.484 5.210 1.499 5.241 1.484
    lz4 1 4 enwik8 53510696 bytes 0.226 0.124 0.226 0.124 0.263 0.125
    lz5 1 4 enwik8 71753612 bytes 0.303 0.171 0.288 0.203 0.294 0.171
    zhuff 0 4 enwik8 37053030 bytes 0.616 0.218 0.632 0.187 0.663 0.187
    lzham 0 4 enwik8 (lzham allow only 3 threads at this system)

    Same batch file but threads changed in 6:
    rar 1 6 enwik8 40565271 bytes 2.336 0.781 2.297 0.781 2.302 0.781
    freearc 1 6 enwik8 38003873 bytes 1.490 0.828 1.474 0.827 1.443 0.828
    lzturbo 10 6 enwik8 52740824 bytes 0.473 0.218 0.508 0.187 0.458 0.211
    zstandard 1 6 enwik8 40859486 bytes 0.592 0.281 0.640 0.281 0.613 0.234
    brotli 1 6 enwik8 42456665 bytes 0.773 0.428 0.790 0.412 0.743 0.422
    qpress 1 6 enwik8 54294428 bytes 0.431 0.343 0.428 0.374 0.506 0.359
    7zip 1 6 enwik8 34707520 bytes 5.365 1.500 5.459 1.531 5.334 1.578
    zip 1 6 enwik8 37907740 bytes 2.646 0.671 2.584 0.641 2.692 0.640
    bzip2 1 6 enwik8 33253096 bytes 9.459 1.531 9.489 1.546 9.428 1.531
    xz 1 6 enwik8 34707456 bytes 5.381 1.515 5.350 1.500 5.412 1.547
    lz4 1 6 enwik8 53510696 bytes 0.331 0.187 0.315 0.187 0.324 0.187
    lz5 1 6 enwik8 71753612 bytes 0.319 0.249 0.330 0.234 0.396 0.234
    zhuff 0 6 enwik8 37053030 bytes 0.818 0.297 0.753 0.281 0.727 0.266
    lzham 0 6 enwik8 (lzham allow only 3 threads at this system)

    Tested at 6 core 12 threads CPU.

    Free RAM DISK software used to create the six test drives:
    https://www.softperfect.com/products/ramdisk/
    Last edited by Sportman; 21st September 2016 at 15:35.

Similar Threads

  1. Benchmarking Entropy Coders
    By dnd in forum Data Compression
    Replies: 194
    Last Post: 22nd February 2020, 00:02
  2. fast opensource multithread stidn packer
    By fcorbelli in forum Data Compression
    Replies: 5
    Last Post: 2nd May 2016, 20:48
  3. Benchmarking fast LZ compressors
    By dnd in forum Data Compression
    Replies: 1
    Last Post: 15th June 2015, 14:12
  4. Archiver benchmarking framework
    By Bulat Ziganshin in forum Data Compression
    Replies: 11
    Last Post: 8th February 2013, 19:46
  5. Replies: 4
    Last Post: 17th March 2008, 21:19

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •