Page 16 of 16 FirstFirst ... 6141516
Results 451 to 479 of 479

Thread: ECT, an file optimizer with fast zopfli-like deflate compression

  1. #451
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    919
    Thanks
    57
    Thanked 113 Times in 90 Posts
    Fixed this with ppx2 completely forgot about this godlike tool
    Last edited by SvenBent; 2nd May 2020 at 07:33.

  2. #452
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    919
    Thanks
    57
    Thanked 113 Times in 90 Posts
    I'm doing another big task of png optimizations and just though id share some results, now non of these are timed. For me it was more to see if we still need other tools for ultimative compresion

    The list is a CHRONOLOGICAL order. its not individual compression iterations to compare
    75 png files comic/drawing nature (Maps) so not reduce collors but bigger than 256
    Code:
    ect -9 -strip --allfilters-b    34.5 MB (36,229,177 bytes)
    pngout /f6              34.5 MB (36,229,177 bytes
    pngout /f5              34.5 MB (36,229,177 bytes)
    pngout /f6 /b512        34.5 MB (36,229,177 bytes)
    pngout /f6 /b2048       34.5 MB (36,229,177 bytes)
    pngout /f6 /b8192       34.5 MB (36,229,177 bytes)
    PNGOut /r (200 trials)  34.5 MB (36,229,098 bytes)
    Deflopt                 34.5 MB (36,229,034 bytes)
    Defluff/deflopt/huffmix 34.5 MB (36,229,015 bytes)

    - Running a simple pngout with original filter or its own mixed filter did nothing for size
    - Running different larger chunks sizes did nothing
    - However Running 200 pngout /r trials on each file had improvements on some of the file.. Showind that pngout /r can sometimes still beats ECT -9 -strip --allfilters-b
    I'm now running a new comparison where its running /r / force iteration using huffmix and deflopt and defluff in the mix for each iteration it will probably be done tomorrow

    I have 2 other sets of png files that I will be testing on as wel but they are still in ECT -9 -strip --allfilters-b phase

    If you have any tools you would like me to try to see if they can squeze more compression after ECT let me know and i'll give it a shoot
    Last edited by SvenBent; 2nd May 2020 at 07:10.

  3. Thanks:

    Krishty (6th May 2020)

  4. #453
    Member
    Join Date
    Mar 2016
    Location
    USA
    Posts
    56
    Thanks
    7
    Thanked 23 Times in 15 Posts
    @SvenBent if you use this branch with options: -99999 -strip --allfilters-b --iterations=0 --stagnations=2000 --ultra=1 do you still have any files that gain improvement with PNGOut? (I still expect a bit of improvement with deflopt/defluff/huffmix).

  5. #454
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    919
    Thanks
    57
    Thanked 113 Times in 90 Posts
    Quote Originally Posted by MegaByte View Post
    @SvenBent if you use this branch with options: -99999 -strip --allfilters-b --iterations=0 --stagnations=2000 --ultra=1 do you still have any files that gain improvement with PNGOut? (I still expect a bit of improvement with deflopt/defluff/huffmix).
    I haven't tested and im not sure i want to test it. its already taking days to go jut with -9 --allfilters-b
    is stagnation the threshold amount of trials with no gains, to stop filter testing ?


    This is still just on going testing but here is another test set of png. (few bigger files)
    Code:
    ect -9 -strip --allfilters-b    112 MB (117,451,983 bytes)
    Pngout /r Huffmix (200 trials)  112 MB (118,242,599 bytes) individual files all bigger. deleted
    
    ect -9 -strip --allfilters-b    112 MB (117,451,983 bytes)
    Deflopt /b                      112 MB (117,451,983 bytes) Firs time ive seen deflopt do nothing
    Defluff/deflopt/huffmix         112 MB (117,451,900 bytes)
    This is the fist time i have not seen deflopt not do anyhing at all on a set of files
    pngout/r huffmix is made with /force so thats why it can grow. i haev to force out a pngout generates file otherwise I cant use huffmix
    I am still figuring out an easy way to handle proper pngout huffmix without risking bigger files. i need to make some comparisons in my batch script

  6. #455
    Member
    Join Date
    Nov 2019
    Location
    Moon
    Posts
    31
    Thanks
    8
    Thanked 32 Times in 20 Posts
    I compiled the latest versions of ECT and iterations branch for Windows with folder support (the AVX2 version is also built with LTO, for some reason it shows slightly better results), but this build requires additional DLLs from MinGW and Boost.

    ​I also made a comparison of various PNG optimizers on a set of random public images, the goal was to get the maximum result, but in a reasonable amount of time (I did not use extreme optimization options).
    https://docs.google.com/spreadsheets...gid=1741276444

    ECT and Pingo, and their combination, showed the best overall result.
    ​Pingo showed better results due to color conversion (from 32 to 24 bits?) on large PNG (which did not do all other optimizers), but I'm not sure that it is lossless.

    Examples of large PNG size differences:
    Source: https://i.redd.it/02pm5dl1ipc41.png
    Pingo - https://i.slow.pics/72GfqzO4.png
    ECT - https://i.slow.pics/QDbFoDbW.png

    Source: https://i.redd.it/mwvt7z763ol41.png
    Pingo - https://i.slow.pics/7E5NXFPh.png
    ECT - https://i.slow.pics/q6uXS4eb.png
    Attached Files Attached Files
    Last edited by Scope; 10th June 2020 at 14:44.

  7. Thanks:

    SvenBent (6th May 2020)

  8. #456
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    919
    Thanks
    57
    Thanked 113 Times in 90 Posts
    Quote Originally Posted by Scope View Post
    ​Pingo showed better results due to color conversion (from 32 to 24 bits?) on large PNG (which did not do all other optimizers), but I'm not sure that it is lossless.
    In the test iran previously ect did also converter32bit (4x8bits) to 24 bits (3x8bits) images unless the alpha channel was needed
    Did you check the image was 100% the same in regards to alpha channel effect as well with pingo ?

    Really nice testing

  9. #457
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    919
    Thanks
    57
    Thanked 113 Times in 90 Posts
    Quote Originally Posted by Scope View Post
    this seesm weird to me
    the source is 24bit
    Both ect and pingo are 48bits for some reason, without knowing to much about your test methology I would say or if this is just a weird case, i would think something went wrong in your testing

    also i jsut confirmed that with 0f5oez7yvi941.png that was 32bit source got conveterd to 24bit with ect -9.
    So if your testing did not show ECT converting 32bit pictures to 24 bit (As claimgi to be the benefits of pingo in your post) when appropriate, your ECT compile might have some issues in it.

    -- edit --
    I ran ect-9 - strip on your first ECT example ( the one with the 48bits version) and it shrunk with 94.65KB out of 7.44MB (1.2427%). im not sure if that is just meta data and your test does not count in for some optimzie removing meta data or not. but 94kb seems like a lot of meta data.. without further information i do believe the tested ect version might be incorrect
    The png was still in 48 bits though


    -- edit ---
    I took your EVT version on you fist examepl that was in 48 bits and saved it as 24 bits.
    I then shutdown my paintshop pro. reopenede both your org file and the now 24bit version of the ect file. and made a arimetihng comparison. all 0 difference on all pixels
    Then I ran ect -9 -strip on it and it came down to 2.61 MB (2,741,225 bytes) down from above 7.44MB

    i dont belvie ther eis any reasone to why you r 24bit source should turn into 48 bits with ect or pingo, so without further information it appears something went wrong in the testing

    --- edit 3--

    I retract my statement. It seems I must have confussed myself
    sorcre is 48bit
    ECT is 48 bits
    pingo is 24 bit

    Looks like ECT does not reduce 48bits to 24 bit even when the resolution is not needed ( it does do 32 to 24 though)
    Maybe we cant get 48 to 24 bits and 64 to 32bit reduction implementet in ect ?

  10. Thanks:

    Krishty (6th May 2020)

  11. #458
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    919
    Thanks
    57
    Thanked 113 Times in 90 Posts
    looking at source and pingo version in FF is can see a clear difference. Pingo looks more faded/grey.

    tried arithmic comparios in psp7.0 they come out to be the same then i realised my PSP might not support 48bit correclt and only do a comparions on the MSB in 24bits ( alpha channels support is also weird at best)
    so i used pngdiff provided to me in another thread
    both source and pingo came out clean.
    I then opened in windows photos and shifted back and forth and there is still a clear difference to my eyes. i do not think its placebok but have not done a propper ABX test

    maybe pngdiff does not support 48bits png ?

    can you confirm or deny that you see the source and pingo image as slightly difference on the mountain picture ?

    -- edit ---
    I used tweakpng to remove CHroma headers and other non critical chunks from the source png file leaving header idat and iend intact. now they look the same in firefox.
    so the difference was maybe due to meta data removal

  12. #459
    Member
    Join Date
    Nov 2019
    Location
    Moon
    Posts
    31
    Thanks
    8
    Thanked 32 Times in 20 Posts
    Quote Originally Posted by SvenBent

    I ran ect-9 - strip on your first ECT example ( the one with the 48bits version) and it shrunk with 94.65KB out of 7.44MB (1.2427%). im not sure if that is just meta data and your test does not count in for some optimzie removing meta data or not. but 94kb seems like a lot of meta data..
    I have sometimes been able to optimize some files a little bit more after reusing ECT if no extreme settings are used.
    But the purpose of the test is to get the maximum efficiency in a reasonable time, with the expectation that a normal person can wait an extra few minutes to optimize a small number of files (but will not wait for hours) or put optimization overnight for a large set and that the result is ready in the morning (but waiting in weeks or months is unacceptable).


    Quote Originally Posted by SvenBent View Post
    Looks like ECT does not reduce 48bits to 24 bit even when the resolution is not needed ( it does do 32 to 24 though)
    Maybe we cant get 48 to 24 bits and 64 to 32bit reduction implementet in ect ?
    I've already received an answer from the author on another topic: https://encode.su/threads/3108-Googl...ll=1#post64836


    Although, I don't know what to do with these files for honest comparison. Remove them? Convert the original so that all optimizers do not convert colors?

  13. #460
    Member
    Join Date
    Apr 2013
    Location
    France
    Posts
    54
    Thanks
    9
    Thanked 19 Times in 16 Posts
    Quote Originally Posted by SvenBent
    maybe pngdiff does not support 48bits png ?
    pngdiff decodes 16 bits/sample PNGs just like web browsers, as 8 bits/sample. the difference you see comes from the iCCP chunk in the original file, deleted by pingo on the user request (aka "-strip" flag). however, the decoded raw rendered pixels by web browsers are *exactly* identical


    Quote Originally Posted by Scope
    Convert the original so that all optimizers do not convert colors?
    there is *no* colors conversion from the encoder. if you load the original image in Firefox, Chromium or whatever web browser, they would convert the image data to 8 bits/sample during the decoding step without any further notice. what would be rendered is the same data that you would find in 8 bits/sample PNG optimized by pingo. if you consider this as lossy, then any 16 bits/sample PNG is lossy decoded by web browsers. FYI, pingo is not the only tool doing this. for example, the cwebp -lossless does this too for WebP


    Quote Originally Posted by Scope
    I don't know what to do with these files for honest comparison
    IMHO, you should define what is the usage context. if those PNG are supposed to be used on web or not. if they do, then pingo did perform a lossless reduction. if not, you could have other issues: unlike the other tool, PNGOUT and OptiPNG are not able to perform 'alpha optimization' (which some could consider as lossy). atm they optimize RGBA samples where this transformations occurs, the comparison would be unfair
    Last edited by cssignet; 29th May 2020 at 14:56. Reason: more exact words

  14. Thanks (2):

    schnaader (7th May 2020),SvenBent (8th May 2020)

  15. #461
    Member
    Join Date
    Nov 2019
    Location
    Moon
    Posts
    31
    Thanks
    8
    Thanked 32 Times in 20 Posts
    Quote Originally Posted by cssignet View Post
    IMHO, you should define what is the usage context. if those PNG are supposed to be used on web or not. if they do, then pingo did perform a lossless reduction. if not, you could have other issues: unlike the other tool, PNGOUT and OptiPNG are not able to perform 'alpha optimization' (which some could consider as lossy). atm they optimize RGBA samples, the comparison would be unfair
    Since this is originally random images from the web and optimization is needed for the same purpose, then most likely I will leave the same results.
    I just needed to understand the reason for this big difference and how it is correct behavior, because all the other optimizers do not produce such a conversion, as opposed to alpha optimization (as it usually does not have such a large benefit of 200-300% of the file size).

  16. #462
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    919
    Thanks
    57
    Thanked 113 Times in 90 Posts
    Quote Originally Posted by cssignet View Post
    pngdiff decodes 16 bits/sample PNGs just like web browsers, as 8 bits/sample. the difference you see comes from the iCCP chunk in the original file, deleted by pingo on the user request (aka "-strip" flag). however, the decoded raw rendered pixels by web browsers are *exactly* identical
    Thank you for the clarification on pngdif.. I though i updated my finding by it was purely from the iCCP chunk I mist have missed in my tired head.
    i used tweakpng ot remove the Iccp chunk and that was exactly the difference I was seeing visually


    When you say it only decodes the 8bits/sample it means it will not detect a difference in the LSB of a 16bit /channel ?
    example
    Picture A has pixels value 37 F6
    Picture B has pixels vale 37 F7.
    pngdif would show 0 difference?

    again I might have reversed my Endian format

  17. #463
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    919
    Thanks
    57
    Thanked 113 Times in 90 Posts
    Quote Originally Posted by Scope View Post
    I have sometimes been able to optimize some files a little bit more after reusing ECT if no extreme settings are used.
    But the purpose of the test is to get the maximum efficiency in a reasonable time, with the expectation that a normal person can wait an extra few minutes to optimize a small number of files (but will not wait for hours) or put optimization overnight for a large set and that the result is ready in the morning (but waiting in weeks or months is unacceptable).
    Nothing wrong with you comparison in my eye on that ground -9 -strip is a pretty fast and reasonable usage method. it also the one i use the most doing comparison on -9-strip- -allfilters-b --pal_sort=120 like I am doing is more for theoretically curiosity, or sometimes i just like to have my computer runs on a project for weeks. buts thats me


    Quote Originally Posted by Scope View Post
    Although, I don't know what to do with these files for honest comparison. Remove them? Convert the original so that all optimizers do not convert colors?
    My opinion on it, depends on the PNG source files
    if all their pixels values are 1111010100000000 1011001000000000 0100010100000000 1000110100000000‬ i see no reason to not converting them to 8bits/channels so they are 11110101 10110010 01000101 100011010.
    but if the values are not having the LSB to be all 8 0bits they should not be reduced and if pingo does that it should be noted or the comparisons, not be included.

    P.S. I might have messed om my endian format but i think my point is clear. if the colors resolution is not needed converting to a lower color resolution should be fine.
    just like we do with 8bits/channel picture to 8bits palette if its less then 256 colors

  18. #464
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    919
    Thanks
    57
    Thanked 113 Times in 90 Posts
    GOt this fancy error

    Decoding error 92: too many pixels, not supported
    Processed 1 file
    Saved 0B out of 1.74MB (0.0000%)

    What is the max size for ect to handle ?

  19. #465
    Member
    Join Date
    Mar 2016
    Location
    USA
    Posts
    56
    Thanks
    7
    Thanked 23 Times in 15 Posts
    That's coming from LodePNG.
      /*multiplication overflow possible further below. Allows up to 2^31-1 pixel
    bytes with 16-bit RGBA, the rest is room for filter bytes.*/
    if(numpixels > 268435455) CERROR_RETURN(state->error, 92);

  20. #466
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    919
    Thanks
    57
    Thanked 113 Times in 90 Posts
    Quote Originally Posted by MegaByte View Post
    That's coming from LodePNG.
      /*multiplication overflow possible further below. Allows up to 2^31-1 pixel
    bytes with 16-bit RGBA, the rest is room for filter bytes.*/
    if(numpixels > 268435455) CERROR_RETURN(state->error, 92);
    Thank you I was at 400mill pixels 8bit rgba though

  21. #467
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    919
    Thanks
    57
    Thanked 113 Times in 90 Posts
    This is my final results i did run 4 different sets of png but they all contains the same type of image. these adre dungeons/room layouts for pen and paper RPG


    Again please note the runs was done sequentially so a instance below another means it was executed on the output from the instance above
    Some place i started over which is indicated with a blank seperation line.

    Code:
    --- 150dpi ---
    ect -9 -strip --allfilters-b    34.5 MB (36,229,177 bytes)
    pngout /f6                      34.5 MB (36,229,177 bytes
    pngout /f5                      34.5 MB (36,229,177 bytes)
    pngout /f6 /b512                34.5 MB (36,229,177 bytes)
    pngout /f6 /b2048               34.5 MB (36,229,177 bytes)
    pngout /f6 /b8192               34.5 MB (36,229,177 bytes)
    PNGOut /r (200 trials)          34.5 MB (36,229,098 bytes)
    Deflopt                         34.5 MB (36,229,034 bytes)
    Defluff/deflopt/huffmix         34.5 MB (36,229,015 bytes)
    
    
    --- Complete Floors ---
    ect -9 -strip --allfilters-b    112 MB (117,451,983 bytes)
    Pngout /r Huffmix (200 trials)  112 MB (118,242,599 bytes) individual files all bigger. deleted)
    
    ect -9 -strip --allfilters-b    112 MB (117,451,983 bytes)
    Deflopt /b                      112 MB (117,451,983 bytes) Firs time ive seen deflopt do nothing)
    Defluff/deflopt/huffmix         112 MB (117,451,900 bytes)
    
    
    --- Full --- 
    ect -9 -strip --allfilters-b    109 MB (114,695,370 bytes)
    Deflopt /b                      109 MB (114,695,370 bytes)
    Defluff/deflopt/huffmix         109 MB (114,695,342 bytes)
    
    ect -9 -strip --allfilters-b    109 MB (114,695,370 bytes)
    Pngout /r Huffmix (200 trials)  109 MB (114,720,180 bytes)
    Mixed files of the above 2      109 MB (114,628,496 bytes)
    
    
    --- no grid ---
    ect -9 -strip --allfilters-b    105 MB (110,486,386 bytes)
    Pngout /r Huffmix (200 trials)  105 MB (110,492,868 bytes)
    Mixed files of the above 2      105 MB (110,404,548 bytes)
    pngout /r was done without creating a new pngout file meaning it would make anew png file if it was smaller hen the one from the previous optimizer
    pngout /r huffmix was done WITH creating new files hence why it sometimes becomes bigger as a hole. its also include delftopt and defluff on each trial


    conclusions in this kind of material :
    pngout is still a contender and can sometimes beat out event ect -9 -strip --allfilters-b
    deflopt/defluff huffmix are still nice tools to have around

  22. Thanks:

    Krishty (13th May 2020)

  23. #468
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    919
    Thanks
    57
    Thanked 113 Times in 90 Posts
    Quote Originally Posted by Krishty View Post
    Yes, please share it!

    If anyone is interested, here is how Papa’s Best Optimizer does it:
    1. ....
    2. defluff
    3. DeflOpt /bk
      • ...
      • because you said it’s best to run it after defluff
    4. if defluff could not shrink the file and DeflOpt printed Number of files rewritten : 0, optimization stops here; else there has been some gain (even single-bit) and it goes back to 3.
      • This is broken in the current version and will be fixed for the next one; missed a 1-B gain on two out of 200 files.
    The next time I have plenty of spare time, I want to check out Huffmix.
    You really should use huffmix when using defluff and deflopt
    I'll how you myt little batch part so show it it should be going

    deflopt %1 /b
    defluff <%1 >%1.fluff.png
    deflopt %1.fluff.png /b
    huffmix %1 %1.fluff.png %1

    Saa bassicly you deflopt you file first.
    then you create a new defluffed files
    you deflop this one as well so both files are deflopt'ed
    Then you use huffmix to find the optimal chunks frorm each file

    The reason you do is this way is becaus deffuf breasks deflopt optimization and can create bigger files.
    so this way with huffman you get the best chunks from both pre and post defluffing




    This is my current new pngout scripts


    Code:
    @echo off
    pngout %1 /force /f6
    
    
    for %%f in (6,0,1,2,3,4,5) do (
    for %%b in (64,96,128,192,256,384,512,768,1024,1536,2048,3072,4096,8192,16384,0) do pngout %1 /f%%f /b%%b /r
    )
    
    
    deflopt %1 /b
    
    
    for /l %%i in (1,1,%2) do (
    pngout %1 %1.tmp.png /force /f6 /ks /r
    deflopt %1.tmp.png /b
    huffmix %1 %1.tmp.png %1
    del %1.tmp.png
    
    defluff <%1 >%1.fluff.png
    deflopt %1.fluff.png /b
    huffmix %1 %1.fluff.png %1
    del %1.fluff.png
    )

    Please not i can create bigger fiels as it forces the fist pngout file.
    but this forcing a pngout fiels has to be done to be able to do all the /r trials with huffxmis
    then later you have to comparee the pngout optimzied files vs ECT or whatever other you use

    but bassically first part tries to find the optimal filters/chunksize combos
    the second parts is just redoign it over and over with random initialization vector nad using huffmix to gain optimal chunkd from each iteration

  24. Thanks:

    Krishty (22nd May 2020)

  25. #469
    Member
    Join Date
    May 2017
    Location
    Germany
    Posts
    86
    Thanks
    55
    Thanked 39 Times in 24 Posts
    Quote Originally Posted by SvenBent View Post
    You really should use huffmix when using defluff and deflopt
    While huffmix works great with pngout /r, I had little success using it on combinations of ECT/DeflOpt/defluff. Details here: https://encode.su/threads/3186-Papa%...5106#post65106

    I should check whether there is a way to use ECT similar to pngout /r, i.e. whether block splits are stable with different parameters …

  26. #470
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    919
    Thanks
    57
    Thanked 113 Times in 90 Posts
    Quote Originally Posted by Krishty View Post
    While huffmix works great with pngout /r, I had little success using it on combinations of ECT/DeflOpt/defluff. Details here: https://encode.su/threads/3186-Papa%...5106#post65106

    I should check whether there is a way to use ECT similar to pngout /r, i.e. whether block splits are stable with different parameters …
    Thank you for the testing i ran into some of the same issues with ECT. it appearsps ECT uses a lot higher number of blocks than pngout
    I reported this issue to caveman in his huffmixthread
    https://encode.su/threads/1313-Huffm...ll=1#post65017

    Personally since Deflopt does never increase size I do not believe its has the biggest effect with huffmix
    but I can ECT + defltop /b mixed with ECT+defluffed+delftop /b, as defluff sometimes increases size.

    i wonder what the huffxmi succes rate is from ECT -9 with pngout /f6 /ks /kp /force on the ect file

  27. #471
    Member
    Join Date
    Nov 2019
    Location
    Moon
    Posts
    31
    Thanks
    8
    Thanked 32 Times in 20 Posts
    Quote Originally Posted by cssignet View Post
    since i do not have the required hardware, and when you have some time, could you try the mp in rc2 41 vs 40? -s0 on ~ 96 files should be enought (if it works - perhaps it would be possible to make it better). thanks
    Unfortunately, I read it late and have already rewritten version 40, but I have done other tests with a different set:
    236 files (845 416 026 bytes)
    Code:
    ect -1 -strip --mt-file *.png
    TotalMilliseconds : 187197,7023
    
    ppx2 -P 8 -L 1 ect.exe -1 -strip "{}" (8 MF)
    TotalMilliseconds : 201311,9329
    
    ect -5 -strip --mt-file *.png
    TotalMilliseconds : 505926,7081
    
    ppx2 -P 8 -L 1 ect.exe -5 -strip "{}" (8 MF)
    TotalMilliseconds : 514763,0062
    
    pingo (41) -s0 -strip *.png
    TotalMilliseconds : 163415,1559
    
    ppx2 -P 8 -L 1 pingo (41) -s0 -strip -nomulti "{}" (8 MF)
    TotalMilliseconds : 142454,0556
    
    pingo (41) -s5 -strip *.png
    ​TotalMilliseconds : 498413,4901
    
    ppx2 -P 8 -L 1 pingo (41) -s5 -strip -nomulti "{}" (8 MF)
    TotalMilliseconds : 398038,6889
    
    pingo (42) -s0 -strip *.png
    TotalMilliseconds : 126019,8598
    
    pingo (42) -s5 -strip *.png
    TotalMilliseconds : 493232,3343

  28. Thanks:

    cssignet (4th July 2020)

  29. #472
    Member
    Join Date
    Apr 2013
    Location
    France
    Posts
    54
    Thanks
    9
    Thanked 19 Times in 16 Posts
    would you please do few more tests, preferably on the same set againts pingo rc2 43 vs proto

    pingo -s0 -strip
    pingo -s0 -strip -mp=8
    proto -s0 -strip
    proto -s0 -strip -mp=8

    it would be nice if you can post the log from pingo + timer/PP64 instead of pshell. thanks
    Last edited by cssignet; 5th July 2020 at 02:09. Reason: remove link

  30. #473
    Member
    Join Date
    Nov 2019
    Location
    Moon
    Posts
    31
    Thanks
    8
    Thanked 32 Times in 20 Posts
    Same set, timer64 (Timer 14.00), pingo (43), proto.
    Hmm, not bad, proto is noticeably faster and more efficient at speed 5 (including -mp=8, and faster, but slightly less efficient at speed 0 (maybe HDD in this configuration can affect, so I tested slower speed 5 too).

    First run:
    Code:
    pingo.exe -s0 -strip *.png 
      pingo - (181.04s):
      -----------------------------------------------------------------
      236 files => 128508.08 KB - (15.57%) saved
      -----------------------------------------------------------------
    Kernel  Time =     0.015 =    0%
    User    Time =     0.000 =    0%
    Process Time =     0.015 =    0%    Virtual  Memory =      9 MB
    Global  Time =   181.088 =  100%    Physical Memory =      8 MB
    
    pingo.exe -s5 -strip *.png 
      pingo - (499.17s):
      -----------------------------------------------------------------
      236 files => 148348.89 KB - (17.97%) saved
      -----------------------------------------------------------------
    Kernel  Time =     0.015 =    0%
    User    Time =     0.000 =    0%
    Process Time =     0.015 =    0%    Virtual  Memory =      9 MB
    Global  Time =   499.209 =  100%    Physical Memory =      8 MB
    
    pingo.exe -s0 -strip *.png -mp=8 
      pingo - (122.29s):
      -----------------------------------------------------------------
      236 files => 128508.08 KB - (15.57%) saved
      -----------------------------------------------------------------
    Kernel  Time =     0.015 =    0%
    User    Time =     0.000 =    0%
    Process Time =     0.015 =    0%    Virtual  Memory =      9 MB
    Global  Time =   122.331 =  100%    Physical Memory =      8 MB
    
    pingo.exe -s5 -strip *.png -mp=8 
      pingo - (487.98s):
      -----------------------------------------------------------------
      236 files => 148348.89 KB - (17.97%) saved
      -----------------------------------------------------------------
    Kernel  Time =     0.031 =    0%
    User    Time =     0.000 =    0%
    Process Time =     0.031 =    0%    Virtual  Memory =      9 MB
    Global  Time =   488.028 =  100%    Physical Memory =      8 MB
    
    proto.exe -s0 -strip *.png
      proto - (97.50s): [ 8 8 ]
      -------------------------------------------------------------------------
      236 files => 128497.20 KB - (15.56%) saved
      -------------------------------------------------------------------------
    Kernel  Time =     0.000 =    0%
    User    Time =     0.015 =    0%
    Process Time =     0.015 =    0%    Virtual  Memory =      9 MB
    Global  Time =    97.553 =  100%    Physical Memory =      8 MB
    
    proto.exe -s5 -strip *.png
      proto - (262.32s): [ 8 8 ]
      -------------------------------------------------------------------------
      236 files => 151476.87 KB - (18.35%) saved
      -------------------------------------------------------------------------
    Kernel  Time =     0.031 =    0%
    User    Time =     0.000 =    0%
    Process Time =     0.031 =    0%    Virtual  Memory =      9 MB
    Global  Time =   262.347 =  100%    Physical Memory =      8 MB
    Second run:
    Code:
    pingo.exe -s0 -strip *.png 
      pingo - (164.35s):
      -----------------------------------------------------------------
      236 files => 128508.08 KB - (15.57%) saved
      -----------------------------------------------------------------
    Kernel  Time =     0.015 =    0%
    User    Time =     0.000 =    0%
    Process Time =     0.015 =    0%    Virtual  Memory =      9 MB
    Global  Time =   164.398 =  100%    Physical Memory =      8 MB
    
    pingo.exe -s5 -strip *.png
      pingo - (504.40s):
      -----------------------------------------------------------------
      236 files => 148348.89 KB - (17.97%) saved
      -----------------------------------------------------------------
    Kernel  Time =     0.031 =    0%
    User    Time =     0.000 =    0%
    Process Time =     0.031 =    0%    Virtual  Memory =      9 MB
    Global  Time =   504.451 =  100%    Physical Memory =      8 MB
    
    pingo.exe -s0 -strip *.png -mp=8
      pingo - (124.96s):
      -----------------------------------------------------------------
      236 files => 128508.08 KB - (15.57%) saved
      -----------------------------------------------------------------
    Kernel  Time =     0.031 =    0%
    User    Time =     0.015 =    0%
    Process Time =     0.046 =    0%    Virtual  Memory =      9 MB
    Global  Time =   124.992 =  100%    Physical Memory =      8 MB
    
    pingo.exe -s5 -strip *.png -mp=8 
      pingo - (489.69s):
      -----------------------------------------------------------------
      236 files => 148348.89 KB - (17.97%) saved
      -----------------------------------------------------------------
    Kernel  Time =     0.000 =    0%
    User    Time =     0.015 =    0%
    Process Time =     0.015 =    0%    Virtual  Memory =      9 MB
    Global  Time =   489.742 =  100%    Physical Memory =      8 MB
    
    proto.exe -s0 -strip *.png 
      proto - (97.89s): [ 8 8 ]
      -------------------------------------------------------------------------
      236 files => 128497.20 KB - (15.56%) saved
      -------------------------------------------------------------------------
    Kernel  Time =     0.015 =    0%
    User    Time =     0.000 =    0%
    Process Time =     0.015 =    0%    Virtual  Memory =      9 MB
    Global  Time =    97.936 =  100%    Physical Memory =      8 MB
    
    proto.exe -s5 -strip *.png
      ​proto - (262.13s): [ 8 8 ]
      -------------------------------------------------------------------------
      236 files => 151476.87 KB - (18.35%) saved
      -------------------------------------------------------------------------
    Kernel  Time =     0.000 =    0%
    User    Time =     0.015 =    0%
    Process Time =     0.015 =    0%    Virtual  Memory =      9 MB
    Global  Time =   262.171 =  100%    Physical Memory =      8 MB

  31. Thanks:

    cssignet (4th July 2020)

  32. #474
    Member
    Join Date
    Apr 2013
    Location
    France
    Posts
    54
    Thanks
    9
    Thanked 19 Times in 16 Posts
    Quote Originally Posted by Scope
    proto is noticeably faster
    here we are. proto is pingo, just with fair comparison of threading. thanks to your test, i would fix that later

    Quote Originally Posted by Scope
    slightly less efficient at speed 0
    just chunks removal, this would be 'fixed'

    now, if you are still up for it, the last trial: proto -s0 -strip, on the set you have test (~900 files) on your benchmark (heuristic test), and a comparison if possible with ECT -1 -strip. thanks!

  33. #475
    Member
    Join Date
    Nov 2019
    Location
    Moon
    Posts
    31
    Thanks
    8
    Thanked 32 Times in 20 Posts
    Quote Originally Posted by cssignet View Post

    now, if you are still up for it, the last trial: proto -s0 -strip, on the set you have test (~900 files) on your benchmark (heuristic test), and a comparison if possible with ECT -1 -strip. thanks!
    I did a test on the entire PNG set (after these multithreading changes are in Pingo, I will redo the speed results of all optimizers when I have time).

    498 files (2 053 230 418 bytes)
    Code:
    ect -1 -strip --mt-file *.png 
    Processed 498 files
    Saved 332.32MB out of 1.91GB (16.9717%)
    Kernel  Time =     0.015 =    0%
    User    Time =     0.000 =    0%
    Process Time =     0.015 =    0%    Virtual  Memory =      9 MB
    Global  Time =   342.581 =  100%    Physical Memory =      8 MB
    
    proto -s0 -strip *.png   proto - (213.94s): [ 8 8 ]
      -------------------------------------------------------------------------
      498 files => 419141.53 KB - (20.90%) saved
      -------------------------------------------------------------------------
    Kernel  Time =     0.015 =    0%
    User    Time =     0.000 =    0%
    Process Time =     0.015 =    0%    Virtual  Memory =      9 MB
    Global  Time =   213.986 =  100%    Physical Memory =      8 MB

  34. Thanks:

    cssignet (4th July 2020)

  35. #476
    Member
    Join Date
    Apr 2013
    Location
    France
    Posts
    54
    Thanks
    9
    Thanked 19 Times in 16 Posts
    ​these are the results i expected. i planned changes for rc2 45 as it could require more trials. thanks for your tests, it was very useful

  36. #477
    Member
    Join Date
    Apr 2013
    Location
    France
    Posts
    54
    Thanks
    9
    Thanked 19 Times in 16 Posts
    the changes would be implemented atm (rc2 44). i did not it test widely though

  37. #478
    Member
    Join Date
    Nov 2019
    Location
    Moon
    Posts
    31
    Thanks
    8
    Thanked 32 Times in 20 Posts
    I haven't tested the speed yet, but I've compared the overall size of the same test set (personally, the most important thing for me is maximum efficiency at the slowest speed and I would be interested in an even slower mode like -sb, but designed for large images, and at faster speeds, exchange a little efficiency for higher speed is reasonable)

    498 files (2 053 230 418 bytes)
    Code:
    pingo (25) -s0 -strip
    1 623 898 128
    
    pingo (25) -s5 -strip
    1 574 034 276
    
    pingo (25) -sa -strip
    1 555 294 498
    
    
    pingo (44) -s0 -strip
    1 623 898 128
    
    pingo (44) -s5 -strip
    1 577 248 424
    
    pingo (46) -s5 -strip
    1 572 187 422
    
    pingo (47) -s5 -strip
    1 570 768 051
    
    pingo (44) -sa -strip
    1 555 295 235
    
    pingo (45) -sa -strip
    1 555 294 330
    
    pingo (46) -sa -strip
    1 572 187 422
    
    pingo (47) -sa -strip
    1 555 295 745
    
    proto -s0 -strip
    1 624 029 494
    
    proto -s5 -strip
    1 567 973 360
    
    proto -sa -strip
    2 053 230 418
    Code:
    pingo (48, 49) -s5 -strip
    1 570 768 051
    
    pingo (51) -s5 -strip
    1 567 820 900
    
    pingo (48) -sa -strip
    1 554 442 346
    
    pingo (49) -sa -strip
    1 554 492 020
    
    pingo (50) -sa -strip
    1 554 272 068
    Click image for larger version. 

Name:	PNG Optimization (total time spent).png 
Views:	19 
Size:	29.9 KB 
ID:	7749
    Last edited by Scope; 9th July 2020 at 16:05.

  38. Thanks:

    cssignet (7th July 2020)

  39. #479
    Member
    Join Date
    Apr 2013
    Location
    France
    Posts
    54
    Thanks
    9
    Thanked 19 Times in 16 Posts
    ​do not waste more time on 46, there is an error on profiles. hopefully this would be improved by speed/size in next release

    edit:
    pingo (50) -sa -strip
    1 554 272 068

    pingo (51) -s5 -strip
    1 567 820 900
    thanks for your test! rc3 would have same results
    Last edited by cssignet; 9th July 2020 at 19:15.

  40. Thanks:

    Scope (10th July 2020)

Page 16 of 16 FirstFirst ... 6141516

Similar Threads

  1. defluff - a deflate huffman optimizer
    By jo.henke in forum Data Compression
    Replies: 48
    Last Post: 7th November 2018, 00:04
  2. Intel's fast OSS deflate implementation
    By Bulat Ziganshin in forum Data Compression
    Replies: 16
    Last Post: 23rd May 2016, 17:13
  3. PNG in .ICO file optimizer ?
    By SvenBent in forum Data Compression
    Replies: 9
    Last Post: 21st April 2016, 17:30
  4. deflate, zopfli, lzma intergers og float heavy ?
    By SvenBent in forum The Off-Topic Lounge
    Replies: 2
    Last Post: 23rd September 2015, 15:41
  5. Replies: 3
    Last Post: 14th April 2015, 00:49

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •