Page 3 of 15 FirstFirst 1234513 ... LastLast
Results 61 to 90 of 440

Thread: ECT, an file optimizer with fast zopfli-like deflate compression

  1. #61
    Member Jaff's Avatar
    Join Date
    Oct 2012
    Location
    Dracula's country
    Posts
    103
    Thanks
    115
    Thanked 22 Times in 18 Posts
    g:\BK\FOTO\div\POZE\DESENE>ect -9 *.jp*
    Caution: quantization tables are too coarse for baseline JPEG
    Processed 646 files
    Saved 0B out of 42.45MB (0.0000%)
    for what file is warning ?

  2. #62
    Member
    Join Date
    Aug 2015
    Location
    Urbana, IL
    Posts
    155
    Thanks
    10
    Thanked 161 Times in 89 Posts
    Quote Originally Posted by Jaff View Post
    for what file is warning ?
    Hmm, that's a good question. AFAIK, there is no way to get libjpeg to print the filename. However, this warning doesn't mean that the file will be corrupted and just indicates that 16-bit quantization tables are used.

  3. #63
    Member
    Join Date
    Apr 2011
    Location
    Russia
    Posts
    168
    Thanks
    163
    Thanked 9 Times in 8 Posts
    dado023
    Attached Files Attached Files

  4. Thanks:

    Jaff (27th March 2016)

  5. #64
    Member
    Join Date
    Apr 2011
    Location
    Russia
    Posts
    168
    Thanks
    163
    Thanked 9 Times in 8 Posts
    fhanau
    I want to use your project in my project? Wanted to learn you don't object?

    For this purpose, I wanted to ask you to make the pngwolf version with your zopfli version.
    I after all very much hope to see the zopfli version compatible to the original zopfli version

  6. #65
    Member
    Join Date
    Aug 2015
    Location
    Urbana, IL
    Posts
    155
    Thanks
    10
    Thanked 161 Times in 89 Posts
    I 'll do that eventually, but currently have little time so it will take a while.

  7. Thanks:

    lorents17 (9th March 2016)

  8. #66
    Member
    Join Date
    Mar 2016
    Location
    Croatia
    Posts
    188
    Thanks
    80
    Thanked 13 Times in 12 Posts

    Question

    Quote Originally Posted by lorents17 View Post
    dado023
    Hi Lorents17,

    thank you for the response, but i am getting this error libwinpthread-1.dll is missing
    Click image for larger version. 

Name:	Km4m3DI.png 
Views:	196 
Size:	25.0 KB 
ID:	4145

    Can this be fixed?

    Thank you in advance

  9. #67
    Member
    Join Date
    Apr 2011
    Location
    Russia
    Posts
    168
    Thanks
    163
    Thanked 9 Times in 8 Posts
    dado023
    Attached Files Attached Files

  10. Thanks:

    Jaff (27th March 2016)

  11. #68
    Member
    Join Date
    Mar 2016
    Location
    Croatia
    Posts
    188
    Thanks
    80
    Thanked 13 Times in 12 Posts
    Quote Originally Posted by lorents17 View Post
    dado023
    Hi Lorents17,

    again thank you for replying


    Here are the results:

    Original PNG:
    227,366 bytes

    iCatalyst-master( [2] Xtreme ) PNG: 183,615 bytes

    iCatalyst+ect( [2] Xtreme ) PNG: 192,488 bytes


    It seems iCatalyst alone has better compression ratio, than, when combined with ECT, .....what sorcery did you do when you got better compression ratio ?

    Thank you in advance
    Dado

  12. #69
    Member
    Join Date
    Apr 2011
    Location
    Russia
    Posts
    168
    Thanks
    163
    Thanked 9 Times in 8 Posts
    please send image

  13. #70
    Member
    Join Date
    Mar 2016
    Location
    Croatia
    Posts
    188
    Thanks
    80
    Thanked 13 Times in 12 Posts
    Quote Originally Posted by lorents17 View Post
    please send image
    due to the nature of image, i have sent it to you via forum PM

    Let me know what can be done.

    Thanks
    Dado

  14. #71
    Member
    Join Date
    Apr 2011
    Location
    Russia
    Posts
    168
    Thanks
    163
    Thanked 9 Times in 8 Posts
    dado023, answer to your question

  15. #72
    Member
    Join Date
    Mar 2016
    Location
    Croatia
    Posts
    188
    Thanks
    80
    Thanked 13 Times in 12 Posts
    Quote Originally Posted by lorents17 View Post
    dado023, answer to your question
    i am not sure what do you mean...i don't understand.......i am only interested how to achieve maximum PNG compression.

  16. #73
    Member
    Join Date
    Apr 2011
    Location
    Russia
    Posts
    168
    Thanks
    163
    Thanked 9 Times in 8 Posts
    dado023
    I have forgotten to insert the reference
    http://encode.su/threads/1689-Google...ll=1#post47048
    Here in what the reason of such difference in compression

  17. #74
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    748
    Thanks
    215
    Thanked 281 Times in 164 Posts
    Quote Originally Posted by lorents17 View Post
    dado023
    I have forgotten to insert the reference
    http://encode.su/threads/1689-Google...ll=1#post47048
    Here in what the reason of such difference in compression
    I tried different variations of brotli's rle optimization in zopfli. None of them gave consistent improvements in a large variety of benchmarks.

    While it is true that brotli's version is more advanced (by for example using fixed point cost model instead of just integers), I have tuned it to the way brotli's Huffman code encoding works. Even when I tried to modify it to the Deflate's encoding, I saw mixed performance in benchmarks.

    If you have a benchmark where brotli's optimize-rle wins over zopfli's optimize-rle, please share it with me.

  18. #75
    Member
    Join Date
    Feb 2016
    Location
    USA
    Posts
    41
    Thanks
    27
    Thanked 63 Times in 24 Posts
    Windows binaries as of 83a1fd9 (changelist). Approx 15% faster for PNG than 0.2.1 in my brief testing, with very slightly better compression.
    Attached Files Attached Files

  19. Thanks:

    Jaff (27th March 2016)

  20. #76
    Member
    Join Date
    Aug 2015
    Location
    Urbana, IL
    Posts
    155
    Thanks
    10
    Thanked 161 Times in 89 Posts
    Quote Originally Posted by Malloc Voidstar View Post
    Windows binaries as of 83a1fd9 (changelist). Approx 15% faster for PNG than 0.2.1 in my brief testing, with very slightly better compression.
    I am going to release a new version this week which will be even faster, so you should build your binaries then.

  21. Thanks:

    Jaff (27th March 2016)

  22. #77
    Member
    Join Date
    Aug 2015
    Location
    Urbana, IL
    Posts
    155
    Thanks
    10
    Thanked 161 Times in 89 Posts
    I released version 0.3.
    Changes:
    As always, more speed and compression for PNG.
    PCLMULQDQ-based crc32 function added. This is about 3 times faster than the slicing-by-8 implementation used previously. On gcc you can use -msse4.2 -mpclmul to enable it.
    SSE4.2 and AVX2 optimized progressive encoding on JPEGs. This is about 20% faster.
    Undocumented arithmetic coding support with "--arithmetic" added.
    Several bugs fixed.

  23. Thanks:

    AiZ (27th March 2016)

  24. #78
    Member
    Join Date
    Mar 2016
    Location
    Croatia
    Posts
    188
    Thanks
    80
    Thanked 13 Times in 12 Posts
    Quote Originally Posted by fhanau View Post
    I released version 0.3.
    Changes:
    As always, more speed and compression for PNG.
    PCLMULQDQ-based crc32 function added. This is about 3 times faster than the slicing-by-8 implementation used previously. On gcc you can use -msse4.2 -mpclmul to enable it.
    SSE4.2 and AVX2 optimized progressive encoding on JPEGs. This is about 20% faster.
    Undocumented arithmetic coding support with "--arithmetic" added.
    Several bugs fixed.

    For us end users, is there compiled version for windows 64bit?

  25. #79
    Member
    Join Date
    Feb 2016
    Location
    USA
    Posts
    41
    Thanks
    27
    Thanked 63 Times in 24 Posts
    v0.3 compiled for Windows with msys2 GCC 5.3.0. "AVX2" builds are actually -march=native -mtune=native on Haswell-E, AFAIK any AVX2-capable CPU should be able to run these.
    Attached Files Attached Files

  26. #80
    Member
    Join Date
    Mar 2016
    Location
    Croatia
    Posts
    188
    Thanks
    80
    Thanked 13 Times in 12 Posts
    Quote Originally Posted by Malloc Voidstar View Post
    v0.3 compiled for Windows with msys2 GCC 5.3.0. "AVX2" builds are actually -march=native -mtune=native on Haswell-E, AFAIK any AVX2-capable CPU should be able to run these.
    Thank you Malloc

  27. #81
    Member nikkho's Avatar
    Join Date
    Jul 2011
    Location
    Spain
    Posts
    550
    Thanks
    222
    Thanked 165 Times in 106 Posts
    Quote Originally Posted by fhanau View Post
    I released version 0.3.
    Changes:
    As always, more speed and compression for PNG.
    PCLMULQDQ-based crc32 function added. This is about 3 times faster than the slicing-by-8 implementation used previously. On gcc you can use -msse4.2 -mpclmul to enable it.
    SSE4.2 and AVX2 optimized progressive encoding on JPEGs. This is about 20% faster.
    Undocumented arithmetic coding support with "--arithmetic" added.
    Several bugs fixed.
    Would you please post 0.3 Win32 and Win64 binaries in the releases page?

  28. #82
    Member
    Join Date
    Apr 2011
    Location
    Russia
    Posts
    168
    Thanks
    163
    Thanked 9 Times in 8 Posts
    fhanau
    Whether you could let out pngwolf-zopfli with your zopfli version?

  29. #83
    Member
    Join Date
    May 2009
    Location
    France
    Posts
    98
    Thanks
    13
    Thanked 74 Times in 44 Posts
    Hello,

    fhanau, could you quickly explain differences between presets please ?
    Code:
                     (e7200@2.86GHz)
    enwik8      size     time
    -2      35354277     0m29
    -3      35140806     0m34
    -4      34978611     0m40
    -5      35014164     0m50
    -6      34991292     1m25
    -7      34987354     1m56
    -8      34984430     6m10
    -9      34984430     6m10
    I remember -M1 being the same than -M2 for gzip/png in previous version of ECT.

    Thanks,

    AiZ

  30. #84
    Member
    Join Date
    Jun 2013
    Location
    Sweden
    Posts
    150
    Thanks
    9
    Thanked 25 Times in 23 Posts
    I just compressed some PNG with ADVpng and PNGout and picked the smallest, then i ran ECT 0.3 -9 --allfilters on all files, saved a few % more.

    All original files and all recompressed files was then converted to BMP with faststone viewer.

    Some files had this difference
    00000026: 23 46
    00000027: 2E 5C
    0000002A: 23 46
    0000002B: 2E 5C

    I think it is DPI 300 vs 600. Some original is 600 and after recompressed 300. Is there a limit of 300 dpi in ECT?

    These 4 bytes is the only difference on just some files.

    EDIT: My misstake - as usual - it´s ADVpng and PNGout that does this, ECT keeps 600 dpi.
    Last edited by a902cd23; 23rd April 2016 at 23:36.

  31. #85
    Member
    Join Date
    Apr 2011
    Location
    Russia
    Posts
    168
    Thanks
    163
    Thanked 9 Times in 8 Posts
    del
    Last edited by lorents17; 9th May 2016 at 18:12.

  32. #86
    Member
    Join Date
    Aug 2015
    Location
    Urbana, IL
    Posts
    155
    Thanks
    10
    Thanked 161 Times in 89 Posts
    v0.4 is out.
    In the summer, I will further improve compression and try to get more of this merged into zopfli.

  33. Thanks (3):

    Jaff (22nd May 2016),Malloc Voidstar (22nd May 2016),Stephan Busch (23rd May 2016)

  34. #87
    Member
    Join Date
    Feb 2016
    Location
    USA
    Posts
    41
    Thanks
    27
    Thanked 63 Times in 24 Posts
    Win32/Win64 binaries, with multi-threading and folder support
    Attached Files Attached Files

  35. Thanks (5):

    dado023 (25th June 2016),Jaff (23rd May 2016),lorents17 (22nd May 2016),nikkho (22nd May 2016),Stephan Busch (23rd May 2016)

  36. #88
    Member
    Join Date
    Jun 2013
    Location
    Sweden
    Posts
    150
    Thanks
    9
    Thanked 25 Times in 23 Posts
    To Malloc Voidstar

    I have i7-3770k with win7x64 and i cannot run ECT_x64.exe, program crash. ECT_x86.exe does work. Yes, 3770 does not have AVX2, only AVX!

    After extracting all three exe to ramdisk and copied newly taken photo (P1140977.JPG), commandline was:
    for %a in (ect*.exe) do ( copy P1140977.JPG %a.jpg & %a -9 %a.jpg )

    Tried -1 to.

  37. #89
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,531
    Thanks
    755
    Thanked 674 Times in 365 Posts
    Quote Originally Posted by fhanau View Post
    PCLMULQDQ-based crc32 function added. This is about 3 times faster than the slicing-by-8 implementation used previously.
    i can't believe! slicing-by-16 has the speeds up to 4 GB/s (i.e. 1 cycle/byte), but speed greatly depends on the compiler's ability to optimally arrange instructions. what's your speed with sse implementation?

  38. #90
    Member
    Join Date
    Feb 2016
    Location
    USA
    Posts
    41
    Thanks
    27
    Thanked 63 Times in 24 Posts
    Quote Originally Posted by a902cd23 View Post
    To Malloc Voidstar

    I have i7-3770k with win7x64 and i cannot run ECT_x64.exe, program crash. ECT_x86.exe does work. Yes, 3770 does not have AVX2, only AVX!

    After extracting all three exe to ramdisk and copied newly taken photo (P1140977.JPG), commandline was:
    for %a in (ect*.exe) do ( copy P1140977.JPG %a.jpg & %a -9 %a.jpg )

    Tried -1 to.
    Does this build work?
    Attached Files Attached Files

  39. Thanks:

    nikkho (25th May 2016)

Page 3 of 15 FirstFirst 1234513 ... LastLast

Similar Threads

  1. defluff - a deflate huffman optimizer
    By jo.henke in forum Data Compression
    Replies: 48
    Last Post: 7th November 2018, 01:04
  2. Intel's fast OSS deflate implementation
    By Bulat Ziganshin in forum Data Compression
    Replies: 16
    Last Post: 23rd May 2016, 18:13
  3. PNG in .ICO file optimizer ?
    By SvenBent in forum Data Compression
    Replies: 9
    Last Post: 21st April 2016, 18:30
  4. deflate, zopfli, lzma intergers og float heavy ?
    By SvenBent in forum The Off-Topic Lounge
    Replies: 2
    Last Post: 23rd September 2015, 16:41
  5. Replies: 3
    Last Post: 14th April 2015, 01:49

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •