Results 1 to 21 of 21

Thread: What's wrong with my testing script?

  1. #1
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts

    [Renamed] Testing - code backup

    The problem are inconsistent results, sometimes there's ~2-3s lag added.

    The test set is a snapshot of my currently developed programs, mostly text + version histories (=archives), additionally some exes. Total-1199 files, 25 803 208 bytes, quite a lot of repeated files (build scripts, license etc.).

    I did 4 tests because of mistakes in archivers' switches.
    1:
    Code:
    "qlz -1"               15037169  0.359 
    "qlz -0"               14742577  0.406 
    "qlz -2"               14568944  0.547 
    "qlz -3"               14095700  1.000 
    "FreeArc -m0"          25819371  1.532 
    "FreeArc -m1"          13161354  1.578 
    "FreeArc -mquick"      14476888  1.906 
    "FreeArc -m1 skip"     11459499  2.281 
    "FreeArc -mquick skip" 12759342  2.907 
    "rar -m1"              13713984  3.125 
    "FreeArc -m2"          10030764  3.141 
    "FreeArc -m3"           9453453  7.266 
    "FreeArc -m4"           9195635 11.313 
    "FreeArc -m5"           9145910 16.359 
    "FreeArc -m6"           9144054 16.610 
    "FreeArc -m7"           9143300 16.391 
    "FreeArc -m8"           9143129 16.219 
    "FreeArc -m9"           9143123 16.219 
    "rar -m2"              13175017 14.953
    2:
    Code:
    "qlz -1"               15037169 0.375 
    "qlz -0"               14742577 0.469 
    "qlz -2"               14568944 0.562 
    "qlz -3"               14095700 1.015 
    "FreeArc -m0"          25819371 1.609 
    "FreeArc -m1"          13161354 1.609 
    "FreeArc -m1 skip"     11459499 2.250 
    "FreeArc -mquick"      14476888 2.453 
    "FreeArc -mquick skip" 12759342 2.907 
    "rar -m1"              13713984 3.031 
    "FreeArc -m2"          10030764 3.093 
    "7z -mx=2"             11941046 6.969 
    "7z -mx=1"             11941046 7.062 
    "FreeArc -m3"           9453453 7.406 
    "FreeArc -m4"           9195635 10.219 
    "7z -mx=3"              9988324 10.938 
    "rar -m2"              13175017 11.484 
    "FreeArc -m9"           9143123 12.812 
    "FreeArc -m7"           9143300 12.953 
    "FreeArc -m8"           9143129 16.343 
    "FreeArc -m5"           9145910 16.656 
    "FreeArc -m6"           9144054 16.829
    3:
    Code:
    "qlz -1"               15037169  0.375 
    "qlz -0"               14742577  0.406 
    "qlz -2"               14568944  0.547 
    "qlz -3"               14095700  0.969 
    "FreeArc -mquick"      14476888  1.266 
    "FreeArc -m0"          25819371  1.610 
    "FreeArc -m1"          13161354  1.750 
    "FreeArc -m1 skip"     11459499  2.297 
    "FreeArc -m2"          10030764  3.125 
    "rar -m1"              13713984  3.078 
    "FreeArc -mquick skip" 12759342  3.531 
    "7z -mx=1"             11941046  7.062 
    "7z -mx=2"             11941046  7.094 
    "FreeArc -m3"           9453453  7.687 
    "7z -mx=3"              9988324 10.266 
    "FreeArc -m4"           9195635 11.047 
    "FreeArc -m8"           9143129 12.891 
    "FreeArc -m6"           9144054 13.063 
    "rar -m2"              13175017 13.594 
    "FreeArc -m5"           9145910 14.907 
    "FreeArc -m7"           9143300 16.125 
    "FreeArc -m9"           9143123 16.500
    4:
    Code:
    "qlz -1"               15037169 0.407 
    "qlz -0"               14742577 0.469 
    "qlz -2"               14568944 0.609 
    "NanoZip -cf"          12587260 1.000 
    "qlz -3"               14095700 1.015 
    "NanoZip -cn"          25857726 1.031 
    "FreeArc -m0"          25819371 1.469 
    "NanoZip -cF"          10845954 1.484 
    "FreeArc -m1"          13161354 1.641 
    "FreeArc -mquick"      14476888 1.719 
    "NanoZip -cd"          10184506 1.938 
    "FreeArc -m1 skip"     11459499 2.344 
    "FreeArc -mquick skip" 12759342 2.984 
    "FreeArc -m2"          10030764 3.078 
    "rar -m1"              13713984 3.093 
    "NanoZip -cD"           9816465 3.594 
    "7z -mx=2"             11941046 6.984 
    "7z -mx=1"             11941046 7.125 
    "FreeArc -m3"           9453453 7.297 
    "7z -mx=3"              9988324 10.047 
    "FreeArc -m4"           9195635 10.296 
    "rar -m2"              13175017 11.687 
    "FreeArc -m7"           9143300 12.735 
    "FreeArc -m9"           9143123 12.859 
    "FreeArc -m5"           9145910 12.938 
    "FreeArc -m6"           9144054 13.313 
    "FreeArc -m8"           9143129 16.969 
    "NanoZip -co"           9779789 22.672 
    "NanoZip -cO"           9064783 84.453 
    "NanoZip -cc"           8789846 128.969
    The problem occurs usually with FreeArc -m5..-m9, but also with rar -m2
    BTW FreeArc does great, but NanoZip...
    Last edited by m^2; 21st September 2008 at 18:59.

  2. #2
    Member
    Join Date
    Aug 2008
    Location
    Saint Petersburg, Russia
    Posts
    215
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Did you also include memory switches for each run? Like, -mXXXXm for NZ?

  3. #3
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    No.

  4. #4
    Member
    Join Date
    Aug 2008
    Location
    Saint Petersburg, Russia
    Posts
    215
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Well, did you do that on purpose? Because that may be the key to your problem. I don't remember how much memory does NZ use by default, but it seems like it's far from much enough for it to show good compression results.

  5. #5
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by nanoflooder View Post
    Well, did you do that on purpose? Because that may be the key to your problem. I don't remember how much memory does NZ use by default, but it seems like it's far from much enough for it to show good compression results.
    So you think that NZ results are poor? Because I find them extraordinary. cf cF cd and cD are give the smallest results when compared to equally fast / faster compressors. If you say that it may be even faster, I'll try to tweak it later.

    And what I call problem is that tests don't give repeatable results.

  6. #6
    Member
    Join Date
    May 2008
    Location
    England
    Posts
    325
    Thanks
    18
    Thanked 6 Times in 5 Posts
    Some variance is to be expected. But even a slight bit of disk access from other tasks can screw the results. I'm guessing nothing else is running when these tests are in progress? ie not even moving the mouse etc? i turn off eveything that isn't needed, esp virus checkers as they'll hurt the performance a lot potentially. I even have an absolute bare bones install of XP so i know nothing will interfere with my tests, most services disabled, no stupid tasks like crappy sound driver apps that system tray etc. Only thing i'll have running is Directory Opus which will be running my test scripts and ATI Tray Tools. All visual crap turned off(set to best performance in the system properties, no swap file(disk thrashing will shaft your results if it starts paging anything out)

  7. #7
    Member
    Join Date
    May 2008
    Location
    Antwerp , country:Belgium , W.Europe
    Posts
    487
    Thanks
    1
    Thanked 3 Times in 3 Posts
    Quote Originally Posted by m^2 View Post
    So you think that NZ results are poor? Because I find them extraordinary. cf cF cd and cD are give the smallest results when compared to equally fast / faster compressors. If you say that it may be even faster, I'll try to tweak it later.
    For highly redundant data you might try :
    Code:
    nz a -cD m1.5g ...  (1,5Gb memory, you can try even higher settings)
    arc a -mexe+delta+lzma:1gb:ht4:128:mc128 (also try higher mc-values, like mc256 or mc512 and higher fb-values : 192 even 273 instead of 128))
    And what I call problem is that tests don't give repeatable results.
    Are these "wall" times or process times ?
    Better use process times, measured by Igor's TIMER.
    How much RAM does your system have ?
    What is the largest available consistent memory block you have before you start testing FA ?
    Last edited by pat357; 6th September 2008 at 16:19.

  8. #8
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Did some testing today.
    Machine: Pentium D 805@2.66 Ghz, 1 GB RAM, XP x64.
    Code:
    "qlz -1"                                     15037169   0.359
    "qlz -0"                                     14742577   0.390
    "4x4 1"                                      17555999   0.438 
    "thor e1"                                    15002518   0.453 
    "4x4 2"                                      14160760   0.500 
    "qlz -2"                                     14568944   0.547
    "4x4 3"                                      13164904   0.547 
    "thor e2"                                    14127036   0.672 
    "NanoZip -cn"                                25857726   0.656
    "4x4 1 -p128"                                17555999   0.781 
    "thor e3"                                    12622556   0.844 
    "NanoZip -cf"                                12587260   0.922
    "NanoZip -cf -m700m"                         12587260   0.938 
    "qlz -3"                                     14095700   0.969 
    "slug"                                       12092726   1.110  
    "FreeArc -m0"                                25819371   1.141 
    "4x4 4"                                      10988868   1.437 
    "NanoZip -cF"                                10845954   1.438  
    "NanoZip -cF -m700m"                         10845954   1.438 
    "FreeArc -m1"                                13161354   1.453   
    "FreeArc -mquick"                            14476888   1.656 
    "NanoZip -cd -m700m"                         10184506   1.860 
    "NanoZip -cd"                                10184506   1.890 
    "4x4 5"                                      10351629   1.937 
    "FreeArc -m1 -ms"                            11459499   2.203  
    "thor e5"                                    13112840   2.375 
    "Tornado c2 x1"                              11343994   2.438 
    "Tornado c1 x1"                              12615810   2.453 
    "Tornado c4 x1"                              10300248   2.766 
    "Tornado c3 x1"                              10475534   2.813 
    "FreeArc -mquick -ms"                        12759342   2.844  
    "rar -m1"                                    13713984   2.984  
    "FreeArc -m2"                                10030764   3.016  
    "thor e4"                                    10584124   3.219 
    "4x4 6"                                      10169667   3.351 
    "PAQ8p -0"                                   25860111   3.406
    "XCOPY"                                      25819371   3.469
    "NanoZip -cD -m700m"                          9816465   3.500 
    "NanoZip -cD"                                 9816465   3.516   
    "Tornado c1 x2"                              12696681   3.750 
    "Tornado c2 x2"                              11402122   3.813 
    "Tornado c3 x2"                              10519150   4.157 
    "Tornado c4 x2"                              10345153   4.172 
    "4x4 1 -p1024"                               17555999   4.234 
    "4x4 7"                                      10030889   5.500 
    "7z -mx=2"                                   11941046   6.969  
    "7z -mx=1"                                   11941046   6.985
    "FreeArc -m3"                                 9453453   7.266  
    "4x4 2 -p1024"                               14160760   7.891 
    "4x4 1t"                                     11057086   8.172 
    "4x4 3 -p1024"                               13164904   8.329 
    "4x4 8"                                       9666401   8.875 
    "bliz f"                                     10846485   9.235 
    "7z -mx=3"                                    9988324  10.015 
    "FreeArc -m4"                                 9195635  10.219 
    "7z -mx=4"                                    9988324  10.312 
    "4x4 3t"                                     10957522  11.079
    "7z -mx=6"                                    9464474  11.312 
    "rar -m2"                                    13175017  11.312 
    "7z -mx=5"                                    9464474  11.547 
    "4x4 4 -p1024"                               10988732  11.562 
    "CTXf -mq"                                   12968143  11.625 
    "bliz c"                                     10806183  11.797 
    "CTXf -mf"                                   12763662  12.188 
    "FreeArc -m5"                                 9145910  12.641 
    "FreeArc -m7"                                 9143300  12.735 
    "FreeArc -m9"                                 9143123  12.812 
    "FreeArc -m8"                                 9143129  12.891
    "rar -m3"                                    12933414  12.953 
    "7z -mx=8"                                    9396262  13.000 
    "FreeArc -m6"                                 9144054  13.063 
    "7z -mx=7"                                    9441597  13.078 
    "m1"                                         13441371  13.547
    "rar -m4"                                    12747964  13.750 
    "4x4 2t"                                     10943913  14.015 
    "CTXf -mq -b"                                11394873  14.125 
    "rar -m5"                                    12730870  14.172 
    "4x4 5 -p1024"                               10352191  14.406 
    "4x4 4t"                                     10494034  14.579 
    "CTXf -mf -b"                                11018242  14.750 
    "4x4 9"                                       9397328  16.500 
    "4x4 10"                                      9377747  20.546 
    "NanoZip -co"                                 9779789  22.672 
    "ccm 0"                                       9371713  23.469 
    "SBC -m1 -b63 -ha"                           11797160  24.750
    "rzm"                                         9139279  43.437 
    "FreeArc -mexe+delta+lzma:1gb:ht4:128:mc128"  9218779  67.703 
    "FreeArc -mexe+delta+lzma:1gb:ht4:192:mc128"  9216363  69.734 
    "NanoZip -cO"                                 9064783  84.453 
    "FreeArc -mexe+delta+lzma:1gb:ht4:128:mc256"  9224229 124.032 
    "FreeArc -mexe+delta+lzma:1gb:ht4:192:mc256"  9222238 126.563 
    "NanoZip -cc"                                 8789846 128.969 
    "FreeArc -mexe+delta+lzma:1gb:ht4:128:mc512"  9259001 237.906 
    "FreeArc -mexe+delta+lzma:1gb:ht4:192:mc512"  9258881 241.531 
    "PAQ8p -1"                                   10232338 633.125 
    "PAQ8p -2"                                    9955560 646.235 
    "PAQ8p -3"                                    9063162 651.203 
    
    4x4 0.2a
    7z 4.57
    Blizzard 0.24b
    ccm 1.30c
    FreeArc 0.5a-230608
    m1 0.0
    NanoZip 0.04
    qlz 1.31
    rar 3.80b4
    RZM 0.07h
    SBC 0.970
    slug 1.27b
    THOR 0.96 alpha
    Tornado 0.4
    All tests done at least 3 times, the best result showed(*).
    Compilation of the best results:
    Code:
    "qlz -1"               15037169   0.359
    "qlz -0"               14742577   0.390
    "4x4 2"                14160760   0.500 
    "4x4 3"                13164904   0.547
    "thor e3"              12622556   0.844 
    "NanoZip -cf"          12587260   0.922
    "slug"                 12092726   1.110   
    "4x4 4"                10988868   1.437 
    "NanoZip -cF"          10845954   1.438   
    "NanoZip -cd -m700m"   10184506   1.860  
    "FreeArc -m2"          10030764   3.016  
    "NanoZip -cD -m700m"    9816465   3.500  
    "FreeArc -m3"           9453453   7.266  
    "FreeArc -m4"           9195635  10.219  
    "FreeArc -m5"           9145910  12.641
    "FreeArc -m7"           9143300  12.735 
    "FreeArc -m9"           9143123  12.812 
    "rzm"                   9139279  43.437 
    "NanoZip -cO"           9064783  84.453 
    "NanoZip -cc"           8789846 128.969
    You may notice some unusual entries like "4x4 2 -p1024" - compression with 1024 threads. I wanted to see how does it affect compression ratio...and it has almost no effect, does 4x4 use slicing data into p parts?
    Quote Originally Posted by Intrinsic View Post
    Some variance is to be expected. But even a slight bit of disk access from other tasks can screw the results. I'm guessing nothing else is running when these tests are in progress? ie not even moving the mouse etc? i turn off eveything that isn't needed, esp virus checkers as they'll hurt the performance a lot potentially. I even have an absolute bare bones install of XP so i know nothing will interfere with my tests, most services disabled, no stupid tasks like crappy sound driver apps that system tray etc. Only thing i'll have running is Directory Opus which will be running my test scripts and ATI Tray Tools. All visual crap turned off(set to best performance in the system properties, no swap file(disk thrashing will shaft your results if it starts paging anything out)
    There was no AV or other services that could look at archivers' hands, nothing CPU intensive and nothing that I expect to write to disk, but the system wasn't clean. Firefox, Miranda, Total Commander, TextPad (no saves/autosaves during tests). I was moving my mouse.
    I know that some variance is usual, but this is really a lot, I really don't think that's it. But I have nothing to loose, I'll try a clean OS, maybe tomorrow.
    Quote Originally Posted by pat357 View Post
    For highly redundant data you might try :
    Code:
    nz a -cD m1.5g ...  (1,5Gb memory, you can try even higher settings)
    arc a -mexe+delta+lzma:1gb:ht4:128:mc128 (also try higher mc-values, like mc256 or mc512 and higher fb-values : 192 even 273 instead of 128))
    Are these "wall" times or process times ?
    Better use process times, measured by Igor's TIMER.
    How much RAM does your system have ?
    What is the largest available consistent memory block you have before you start testing FA ?
    LZMA doesn't work well here...too much text, most of the rest is not compressible anyway.

    I didn't test fb 273, I tested 276 instead (mistake ) and got errors. It's way too slow to be practical and not expected to perform well anyway, so I'll skip this.
    Nanozip picks memory well, results are exactly the same.

    What are "wall" times? I use Igor's timer.
    How can I know what is the largest consistent memory block in my system ?? And how does it matter? Isn't Windows' virtual memory smart enough to allocate inconsistent blocks? All I can tell is that I have ~600-700 MB free and I remember seeing one archiver claiming to allocate ~800MB.

    (*)Arc -mquick has one result 1.266, seven ~1.7-1.8 and one 2.4xx. I guess something was wrong with the extreme times and picked the second best.

    Thank you all for suggestions.

  9. #9
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    There was so much to write that I forgot to ask about one effect that I don't understand.
    With all tests, my batches start with compressing data twice with qlz, just to fill system file cache. However, I noticed that if I run XCOPY on the test set, speed of archiving without compression is greatly reduced. With compression - no effect at all.

  10. #10
    Member
    Join Date
    May 2008
    Location
    England
    Posts
    325
    Thanks
    18
    Thanked 6 Times in 5 Posts
    Just remember that any activity that takes focus from a task will draw cpu power away from it, even if it runs on one core while you use another. So i just set stuff running and leave the comp todo it's thing which i watch TV

    And yep it's good you run it at least twice, i think most of us do that too, or even just dump it out to /dev/null(or windows equiv) once then kick it off for real.

    All I can tell is that I have ~600-700 MB free and I remember seeing one archiver claiming to allocate ~800MB.
    If that's the case it'll seriously effect the processing time, as it'll page stuff out of memory to make room. Before i got my new comp i was running at the same amount of free mem on one of my more "busy"(read: full of crap) XP installs, so i only let(where possible) a method use 512mb at max to reducing disk thrashing from paging.
    Last edited by Intrinsic; 7th September 2008 at 02:26.

  11. #11
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by Intrinsic View Post
    Just remember that any activity that takes focus from a task will draw cpu power away from it, even if it runs on one core while you use another. So i just set stuff running and leave the comp todo it's thing which i watch TV
    I really don't believe that such light usage has measurable impact on single threaded compressors, unless there's some HDD usage that I don't know about. But, as I said, I'll do some tests. My dog is gonna like it.

    Quote Originally Posted by Intrinsic View Post
    If that's the case it'll seriously effect the processing time, as it'll page stuff out of memory to make room. Before i got my new comp i was running at the same amount of free mem on one of my more "busy"(read: full of crap) XP installs, so i only let(where possible) a method use 512mb at max to reducing disk thrashing from paging.
    Good point, I'll care more in the future.
    The archiver which used this memory was CCM with -6, (-7, 1.2 GB works too), not benchmarked, probably nothing on the list is affected.

  12. #12
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Some FreeArc tests (I did it yesterday, not clean machine yet..):
    Code:
    "FreeArc -m0"                                25819371   1.141 
      
    "FreeArc -m1 -ms"                            11459499   2.203
    "FreeArc -m1 -ma-2 -ms"                      11459499   2.125 
    "FreeArc -m1 -ma- -ms"                       11543101   2.140  
    "FreeArc -m1"                                13161354   1.453 
    "FreeArc -m1 -ma-2"                          13161354   1.406 
    "FreeArc -m1 -ma-"                           13214237   1.437   
    
    "FreeArc -m2"                                10030764   3.016 
    "FreeArc -m2 -ma-2 -ms"                      10084297   3.719 
    "FreeArc -m2 -ms"                            10107256   4.093 
    "FreeArc -m2 -ma-2"                          10121896   2.969 
    "FreeArc -m2 -ma- -ms"                       10168296   3.734   
    "FreeArc -m2 -ma-"                           10203831   2.984  
    
    "FreeArc -m3 -ms"                             9445514   8.453 
    "FreeArc -m3"                                 9453453   7.266 
    "FreeArc -m3 -ma-2 -ms"                       9500774   8.532 
    "FreeArc -m3 -ma- -ms"                        9507177   8.547 
    "FreeArc -m3 -ma-2"                           9519822   7.781 
    "FreeArc -m3 -ma-"                            9525659   7.843  
    
    "FreeArc -m4"                                 9195635  10.219 
    "FreeArc -m4 -ms"                             9210841  10.188 
    "FreeArc -m4 -ma-2 -ms"                       9272311  10.875 
    "FreeArc -m4 -ma-2"                           9272529  10.844 
    "FreeArc -m4 -ma- -ms"                        9277863  10.877
    "FreeArc -m4 -ma-"                            9278056  10.844  
    
    "FreeArc -m5 -ms"                             9135677  13.141 
    "FreeArc -m5"                                 9145910  12.641 
    "FreeArc -m5 -ma-2"                           9172643  12.140
    "FreeArc -m5 -ma- -ms"                        9174156  12.344
    "FreeArc -m5 -ma-2 -ms"                       9177111  12.281 
    "FreeArc -m5 -ma-"                            9178013  12.360
     
    "FreeArc -m6 -ms"                             9133877  13.188 
    "FreeArc -m6"                                 9144054  13.063
    "FreeArc -m6 -ma-2"                           9170657  13.906 
    "FreeArc -m6 -ma- -ms"                        9172243  12.657
    "FreeArc -m6 -ma-2 -ms"                       9175141  12.593
    "FreeArc -m6 -ma-"                            9176091  14.171
    
    "FreeArc -m7 -ms"                             9133065  13.297 
    "FreeArc -m7"                                 9143300  12.735 
    "FreeArc -m7 -ma-2"                           9169918  14.391 
    "FreeArc -m7 -ma- -ms"                        9171488  12.515
    "FreeArc -m7 -ma-2 -ms"                       9174396  12.938 
    "FreeArc -m7 -ma-"                            9175335  14.375 
    
    "FreeArc -m8 -ms"                             9132916  13.500 
    "FreeArc -m8"                                 9143129  12.891
    "FreeArc -m8 -ma-2"                           9169755  14.375 
    "FreeArc -m8 -ma- -ms"                        9171332  12.547
    "FreeArc -m8 -ma-2 -ms"                       9174233  13.281 
    "FreeArc -m8 -ma-"                            9175177  14.203 
    
    "FreeArc -m9 -ms"                             9132900  13.203 
    "FreeArc -m9"                                 9143123  12.812 
    "FreeArc -m9 -ma-2"                           9169761  14.187 
    "FreeArc -m9 -ma- -ms"                        9171322  12.610
    "FreeArc -m9 -ma-2 -ms"                       9174243  13.312 
    "FreeArc -m9 -ma-"                            9175162  14.172 
    
    "FreeArc -mx -ms"                             9132900  13.266 
    "FreeArc -mx"                                 9144779  19.610 
    "FreeArc -mx -ma-2"                           9169761  14.188
    "FreeArc -mx -ma- -ms"                        9171322  12.625
    "FreeArc -mx -ma-2 -ms"                       9174243  13.453 
    "FreeArc -mx -ma-"                            9175162  14.203 
    
    "FreeArc -max -ms"                            9049012  20.593 
    "FreeArc -max"                                9060380  22.125
    "FreeArc -max -ma-2"                          9085152  20.860  
    "FreeArc -max -ma- -ms"                       9088168  19.985
    "FreeArc -max -ma-2 -ms"                      9089205  19.703 
    "FreeArc -max -ma-"                           9091297  21.156 
    
    "FreeArc -mexe+delta+lzma:1gb:ht4:128:mc128"  9218779  67.703 
    "FreeArc -mexe+delta+lzma:1gb:ht4:128:mc256"  9224229 124.032 
    "FreeArc -mexe+delta+lzma:1gb:ht4:128:mc512"  9259001 237.906 
    "FreeArc -mexe+delta+lzma:1gb:ht4:192:mc128"  9216363  69.734 
    "FreeArc -mexe+delta+lzma:1gb:ht4:192:mc256"  9222238 126.563 
    "FreeArc -mexe+delta+lzma:1gb:ht4:192:mc512"  9258881 241.531 
    
    "FreeArc -mquick"                            14476888   1.656 
    "FreeArc -mquick -ms"                        12759342   2.844
    I merged old and new results.
    Notation: -ma-2 is -ma- switch with a tweaked arc.groups (general tweaks, not for this particular test set) that I posted in the FreeArc topic.
    Conclusions:
    - -ms improves compression ratio here, except for -ma-2 in modes 5+. That's because -ma-2 correctly detects some sqx archives that happen to be compressible further.
    - -ma- and -ma-2 are equally fast and ~0.5s faster than detection by content.
    - -ma-2 has slightly better compression ratio than -ma- (<1%), automatic detection works best, but the difference is slight too.
    - in faster modes -ms slows compression down. In slower - speeds up.
    - -m5..9/x are almost the same.

    Updated list of the most efficient programs / modes:
    Code:
    "qlz -1"                15037169   0.359
    "qlz -0"                14742577   0.390
    "4x4 2"                 14160760   0.500 
    "4x4 3"                 13164904   0.547
    "thor e3"               12622556   0.844 
    "NanoZip -cf"           12587260   0.922
    "slug"                  12092726   1.110   
    "4x4 4"                 10988868   1.437 
    "NanoZip -cF"           10845954   1.438   
    "NanoZip -cd -m700m"    10184506   1.860  
    "FreeArc -m2 -ma-2"     10121896   2.969 
    "FreeArc -m2"           10030764   3.016  
    "NanoZip -cD -m700m"     9816465   3.500  
    "FreeArc -m3"            9453453   7.266 
    "FreeArc -m3 -ms"        9445514   8.453  
    "FreeArc -m4 -ms"        9210841  10.188 
    "FreeArc -m4"            9195635  10.219 
    "FreeArc -m5 -ma-2"      9172643  12.140  
    "FreeArc -m7 -ma- -ms"   9171488  12.515
    "FreeArc -m8 -ma- -ms"   9171332  12.547 
    "FreeArc -m5"            9145910  12.641
    "FreeArc -m7"            9143300  12.735 
    "FreeArc -m9"            9143123  12.812
    "FreeArc -m5 -ms"        9135677  13.141  
    "FreeArc -m6 -ms"        9133877  13.188
    "FreeArc -m9 -ms"        9132900  13.203 
    "FreeArc -max -ma-2 -ms" 9089205  19.703 
    "FreeArc -max -ma- -ms"  9088168  19.985
    "FreeArc -max -ms"       9049012  20.593  
    "NanoZip -cc"            8789846 128.969

  13. #13
    Member
    Join Date
    May 2008
    Location
    England
    Posts
    325
    Thanks
    18
    Thanked 6 Times in 5 Posts
    Yeah i only used CCM with the 274Mb used option(-4 i think?) Otherwise it'll grind away heh.

  14. #14
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Clean machine didn't help :/
    I did most tests 3 more times and additionally some more:
    Code:
    "CCM 0"       9371713  22.641 
    "CCM 1"       9293769  23.266 
    "CCM 2"       9185369  23.563 
    "CCM 3"       9162288  24.172
    "CCM 4"       9148813  24.781 
    "CCM 5"       9140366  25.422 
    "CCM 6"       9135885  26.094
    "CCMX 0"      9372462  26.250  
    "CCMX 1"      9290740  27.094 
    "CCMX 2"      9179047  27.984 
    "CCMX 3"      9152385  29.063 
    "CCMX 4"      9134386  29.672 
    "CCMX 5"      9121118  30.329 
    "CCMX 6"      9112756  31.000 
    
    "SQX -m1"    10322898   8.672 
    "SQX -m2"    10303146   9.250 
    "SQX -m3"    10296038   9.843 
    "SQX -m4"    10112262  20.578 
    "SQX -m5"     9925270  24.843 
    "SQX -mx"     9968430  23.406 
    
    "BriefLZ"    15652257   1.422 
    
    "lzop -1 -F" 15009233   0.640 
    "lzop -1"    15009645   0.672 
    "lzop -2 -F" 15035957   0.703 
    "lzop -2"    15036369   0.656 
    "lzop -3 -F" 15035957   0.640 
    "lzop -3"    15036369   0.656 
    "lzop -4 -F" 15035957   0.625 
    "lzop -4"    15036369   0.671 
    "lzop -5 -F" 15035957   0.641
    "lzop -5"    15036369   0.703 
    "lzop -6 -F" 15035957   0.625 
    "lzop -6"    15036369   0.719
    "lzop -7 -F" 13757820   4.890 
    "lzop -7"    13757820   4.968 
    "lzop -8 -F" 13720012   8.907 
    "lzop -8"    13720012   9.000 
    "lzop -9 -F" 13715518  10.844 
    "lzop -9"    13715518  10.921
    Attached Files Attached Files

  15. #15
    Member
    Join Date
    May 2008
    Location
    Antwerp , country:Belgium , W.Europe
    Posts
    487
    Thanks
    1
    Thanked 3 Times in 3 Posts
    Quote Originally Posted by m^2 View Post
    Nanozip picks memory well, results are exactly the same.
    With only 1GB RAM, NZ with -m1.5g has no sense of course
    NZ will probably use all the available RAM on your system, limited to the largest available consistent memory block. (see below)
    What are "wall" times?
    Wall times are like the "global time" displayed by Igor's timer.
    How can I know what is the largest consistent memory block in my system ??
    Look at http://forall.ru-board.com/egor23/on...al_Memory.html
    There is a tool "memo2g". thats can show you this value.
    (page is in Russian, you can use Google to translate (like I did )
    And how does it matter? Isn't Windows' virtual memory smart enough to allocate inconsistent blocks?
    Unfortunately windows doesn't do this well.
    Lzma with large dictionary (or in case of NZ, BWT with larger blocks) will handle redundant data (duplicate or similar files in this test) much better.
    The maximum value for the dictionary is however limited by the largest available consistent memory block. If your memory gets fragmented, the max dictionary will decrease and you might get worse compression for redundant data. (like your test-data)
    To see the influence of a larger dict on redundant data, you could try the following :
    try to compress 2 identical files from about 100Mb ( a 100Mb .zip, rar or any compressed file will show this effect even better)
    use your favorite archivers to compress these 2 files
    Code:
    QLZ
    RAR -m2 .. -m5 
    7z 
    arc a -mlzma:1gb:ht4:16:mc8
    arc m2 m3  and max
    NZ -cd .. cO with m50m ,m200m, m300m
    CCM 3 and 5

    How did RAR perform compared to the other ?
    What is the ratio form NZ -m50m compared to m300m ?
    Last edited by pat357; 8th September 2008 at 01:42.

  16. #16
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by pat357 View Post
    Wall times are like the "global time" displayed by Igor's timer.
    Process time measures only CPU time, right? I wanted it to be a real life test that takes into account HDD handling too and therefore - I measured global times.

    Quote Originally Posted by pat357 View Post
    Look at http://forall.ru-board.com/egor23/on...al_Memory.html
    There is a tool "memo2g". thats can show you this value.
    (page is in Russian, you can use Google to translate (like I did )

    Unfortunately windows doesn't do this well.
    Sadly, google (altavista too) barely knows russian..and this coupled with my not so good english makes this article hard to understand. But I guess that the issue is not inability to allocate non continuous block of physical memory but lack of free continuous block in process' address space, right?

    Quote Originally Posted by pat357 View Post
    Lzma with large dictionary (or in case of NZ, BWT with larger blocks) will handle redundant data (duplicate or similar files in this test) much better.
    The maximum value for the dictionary is however limited by the largest available consistent memory block. If your memory gets fragmented, the max dictionary will decrease and you might get worse compression for redundant data. (like your test-data)
    To see the influence of a larger dict on redundant data, you could try the following :
    try to compress 2 identical files from about 100Mb ( a 100Mb .zip, rar or any compressed file will show this effect even better)
    use your favorite archivers to compress these 2 files
    Code:
    QLZ
    RAR -m2 .. -m5 
    7z 
    arc a -mlzma:1gb:ht4:16:mc8
    arc m2 m3  and max
    NZ -cd .. cO with m50m ,m200m, m300m
    CCM 3 and 5

    How did RAR perform compared to the other ?
    What is the ratio form NZ -m50m compared to m300m ?
    This test set has just 25MB, so there's no need for huge dictionary.
    Generally, I know what large dictionaries are for, no need to test it. At least with LZ*.
    I don't know CM, I briefly looked at LPAQ sources once and it seemed to handle memory in a totally different way, right?

  17. #17
    Member
    Join Date
    May 2008
    Location
    Antwerp , country:Belgium , W.Europe
    Posts
    487
    Thanks
    1
    Thanked 3 Times in 3 Posts
    Quote Originally Posted by m^2 View Post
    the issue is not inability to allocate non continuous block of physical memory but lack of free continuous block in process' address space, right?
    Indeed, that was my point.
    Have you tried to run "memo2g.exe" ?
    I don't know CM, I briefly looked at LPAQ sources once and it seemed to handle memory in a totally different way, right?
    Larger memory model for CM will also improve compression for redundant data, just like it's for LZ* based compression.

  18. #18
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by pat357 View Post
    Indeed, that was my point.
    Have you tried to run "memo2g.exe" ?

    Larger memory model for CM will also improve compression for redundant data, just like it's for LZ* based compression.
    I wouldn't believe if you said otherwise, I know how much do the biggest PAQ modes use:P
    But does huge amount of memory help on small redundant data?
    memo2g? 1.7 GB/
    Last edited by m^2; 10th September 2008 at 01:14.

  19. #19
    Member
    Join Date
    May 2008
    Location
    Antwerp , country:Belgium , W.Europe
    Posts
    487
    Thanks
    1
    Thanked 3 Times in 3 Posts
    Quote Originally Posted by m^2 View Post
    I wouldn't believe if you said otherwise, I know how much do the biggest PAQ modes use:P
    But does huge amount of memory help on small redundant data?
    I don't think so. Once the whole file fits in the "memory model", increasing memory settings doesn't help anymore for gaining ratio's. Of course the overall-ratio might still increase due better compression of all the data, but this is not specific for "duplicate files".
    Quote Originally Posted by m^2 View Post
    memo2g?
    http://www.haskell.org/bz/memo.7z (I think Bulat has something to do with this little tool)

  20. #20
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by pat357 View Post
    I don't think so. Once the whole file fits in the "memory model", increasing memory settings doesn't help anymore for gaining ratio's. Of course the overall-ratio might still increase due better compression of all the data, but this is not specific for "duplicate files".
    Thanks.

    Quote Originally Posted by pat357 View Post
    http://www.haskell.org/bz/memo.7z (I think Bulat has something to do with this little tool)
    It was meant to be:
    You ask for memo2g result? You got memo2g result.
    1.7 GB.

  21. #21
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    New archivers tested.

    BALZ 1.15:
    Code:
    BALZ e  	9869984	14.640
    BALZ ex 	9798204	24.359
    BCM 0.02:
    Code:
    BCM	11554980	562.312	Tested only once, some variation expected
    BIT 0.6:
    Code:
    BIT fastest	11036333	88.797
    BIT fast	10630360	90.031
    BIT normal	10208519	92.531
    BZP 0.1:
    Code:
    BZP	12541629	3.640
    CMM4 0.2b:
    Code:
    CMM4 11 	10244446	53.281
    CMM4 55  	 9141866	61.422
    Dark 0.51:
    Code:
    Dark	12581652	13.828
    FastLZ 0.1:
    Code:
    FastLZ -2	15479416	0.359
    FastLZ opt -2	15693750	0.359
    FastLZ opt -1	16078104	0.375
    FastLZ -1	16041638	0.390
    LPAQ2:
    Code:
    LPAQ2 1 	10575032	90.313	
    LPAQ2 3 	 9401627	91.719	
    LPAQ2 2 	10298349	92.953	
    LPAQ2 4 	 9306369	95.938	
    LPAQ2 5 	 9200271	102.204	
    LPAQ2 6 	 9171147	107.219	
    LPAQ2 7 	 9155945	112.578
    LZSS 0.01:
    Code:
    LZSS e  	14779580	4.984
    LZSS ex 	14135294	161.796
    LZTurbo 0.92:
    Code:
    LZTurbo -11	14524917	0.328	
    LZTurbo -21	14170363	0.375	
    LZTurbo -31	14640126	0.484	
    LZTurbo -41	12817012	0.828	
    LZTurbo -22	10930618	1.328	
    LZTurbo -12	12066470	1.328	
    LZTurbo -32	11393232	1.406	
    LZTurbo -13	11715839	1.641	
    LZTurbo -23	10588677	1.687	
    LZTurbo -33	11065427	1.765	
    LZTurbo -42	10986725	2.016	
    LZTurbo -43	10656955	2.313	
    LZTurbo -14	11633415	2.390	
    LZTurbo -24	10515376	2.421	
    LZTurbo -34	10995408	2.453	
    LZTurbo -15	11584213	2.985	
    LZTurbo -25	10473965	3.015	
    LZTurbo -44	10583022	3.032	
    LZTurbo -35	10954957	3.125	
    LZTurbo -45	10540015	3.672	
    LZTurbo -16	11554016	4.094	
    LZTurbo -26	10448628	4.250	
    LZTurbo -36	10930302	4.422	
    LZTurbo -46	10512364	4.938	
    LZTurbo -51	13157820	5.250	
    LZTurbo -52	11781629	5.875	
    LZTurbo -17	11534561	6.047	
    LZTurbo -27	10432261	6.313	
    LZTurbo -53	11496804	6.344	
    LZTurbo -37	10914230	6.438	
    LZTurbo -47	10494484	7.000	
    LZTurbo -54	11430794	7.172	
    LZTurbo -55	11393041	7.984	
    LZTurbo -56	11369111	9.094	
    LZTurbo -57	11353115	11.344	
    LZTurbo -18	11518667	14.688	
    LZTurbo -28	10418632	15.532	
    LZTurbo -38	10900408	15.547	
    LZTurbo -48	10477226	16.032	
    LZTurbo -58	11338074	20.344	
    LZTurbo -19	11295390	50.719	
    LZTurbo -29	10318945	53.234	
    LZTurbo -39	10785274	53.781	
    LZTurbo -49	10253181	54.516	
    LZTurbo -59	11056091	58.812
    qpress 0.14 BETA:
    Code:
    qpress -L1	16004979	1.235	
    qpress -L2	15029740	1.547	
    qpress -L3	14743422	2.734	
    qpress -BL1	16004979	7.343	
    qpress -BL2	15029740	7.625	
    qpress -BL3	14743422	8.734
    QUAD 1.12:
    Code:
    QUAD	10398797	9.781
    Updated "The best" list:
    Code:
    LZTurbo -11           14524917   0.328
    LZTurbo -21           14170363   0.375
    4x4 2                 14160760   0.500 
    4x4 3                 13164904   0.547
    LZTurbo -41           12817012   0.828
    thor e3               12622556   0.844 
    NanoZip -cf           12587260   0.922
    slug                  12092726   1.110 
    LZTurbo -22           10930618   1.328
    NanoZip -cF           10845954   1.438  
    LZTurbo -23           10588677   1.687
    NanoZip -cd -m700m    10184506   1.860  
    FreeArc -m2 -ma-2     10121896   2.969 
    FreeArc -m2           10030764   3.016  
    NanoZip -cD -m700m     9816465   3.500  
    FreeArc -m3            9453453   7.266 
    FreeArc -m3 -ms        9445514   8.453  
    FreeArc -m4 -ms        9210841  10.188 
    FreeArc -m4            9195635  10.219 
    FreeArc -m5 -ma-2      9172643  12.140  
    FreeArc -m7 -ma- -ms   9171488  12.515
    FreeArc -m8 -ma- -ms   9171332  12.547 
    FreeArc -m5            9145910  12.641
    FreeArc -m7            9143300  12.735 
    FreeArc -m9            9143123  12.812
    FreeArc -m5 -ms        9135677  13.141  
    FreeArc -m6 -ms        9133877  13.188
    FreeArc -m9 -ms        9132900  13.203 
    FreeArc -max -ma-2 -ms 9089205  19.703 
    FreeArc -max -ma- -ms  9088168  19.985
    FreeArc -max -ms       9049012  20.593  
    NanoZip -cc            8789846 128.969
    Attached Files Attached Files

Similar Threads

  1. FreeArc and Inno Setup script
    By mondragon in forum Data Compression
    Replies: 101
    Last Post: 31st March 2015, 20:13
  2. Help beta testing QuickLZ 1.40 with the new test framework
    By Lasse Reinhold in forum Forum Archive
    Replies: 10
    Last Post: 19th April 2008, 16:16

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •