ECT, an file optimizer with fast zopfli-like deflate compression

[Jaff]

g:\BK\FOTO\div\POZE\DESENE>ect -9 *.jp*
Caution: quantization tables are too coarse for baseline JPEG
Processed 646 files
Saved 0B out of 42.45MB (0.0000%)

for what file is warning ?

[fhanau]

for what file is warning ?

Hmm, that's a good question. AFAIK, there is no way to get libjpeg to print the filename. However, this warning doesn't mean that the file will be corrupted and just indicates that 16-bit quantization tables are used.

[lorents17]

dado023

[lorents17]

fhanau
I want to use your project in my project? Wanted to learn you don't object?

For this purpose, I wanted to ask you to make the pngwolf version with your zopfli version.
I after all very much hope to see the zopfli version compatible to the original zopfli version

[fhanau]

I 'll do that eventually, but currently have little time so it will take a while.

[dado023]

dado023

Hi Lorents17,

thank you for the response, but i am getting this error libwinpthread-1.dll is missing


Can this be fixed?

Thank you in advance

[lorents17]

dado023

[dado023]

dado023

Hi Lorents17,

again thank you for replying


Here are the results:

Original PNG: 227,366 bytes

iCatalyst-master( [2] Xtreme ) PNG: 183,615 bytes

iCatalyst+ect( [2] Xtreme ) PNG: 192,488 bytes


It seems iCatalyst alone has better compression ratio, than, when combined with ECT, .....what sorcery did you do when you got better compression ratio ?

Thank you in advance
Dado

[lorents17]

please send image

[dado023]

please send image

due to the nature of image, i have sent it to you via forum PM

Let me know what can be done.

Thanks
Dado

[lorents17]

dado023, answer to your question

[dado023]

dado023, answer to your question

i am not sure what do you mean...i don't understand.......i am only interested how to achieve maximum PNG compression.

[lorents17]

dado023
I have forgotten to insert the reference
http://encode.su/threads/1689-Google...ll=1#post47048
Here in what the reason of such difference in compression

[Jyrki Alakuijala]

dado023
I have forgotten to insert the reference
http://encode.su/threads/1689-Google...ll=1#post47048
Here in what the reason of such difference in compression

I tried different variations of brotli's rle optimization in zopfli. None of them gave consistent improvements in a large variety of benchmarks.

While it is true that brotli's version is more advanced (by for example using fixed point cost model instead of just integers), I have tuned it to the way brotli's Huffman code encoding works. Even when I tried to modify it to the Deflate's encoding, I saw mixed performance in benchmarks.

If you have a benchmark where brotli's optimize-rle wins over zopfli's optimize-rle, please share it with me.

[Malloc Voidstar]

Windows binaries as of 83a1fd9 (changelist). Approx 15% faster for PNG than 0.2.1 in my brief testing, with very slightly better compression.

[fhanau]

Windows binaries as of 83a1fd9 (changelist). Approx 15% faster for PNG than 0.2.1 in my brief testing, with very slightly better compression.

I am going to release a new version this week which will be even faster, so you should build your binaries then.

[fhanau]

I released version 0.3.
Changes:
As always, more speed and compression for PNG.
PCLMULQDQ-based crc32 function added. This is about 3 times faster than the slicing-by-8 implementation used previously. On gcc you can use -msse4.2 -mpclmul to enable it.
SSE4.2 and AVX2 optimized progressive encoding on JPEGs. This is about 20% faster.
Undocumented arithmetic coding support with "--arithmetic" added.
Several bugs fixed.

[dado023]

I released version 0.3.
Changes:
As always, more speed and compression for PNG.
PCLMULQDQ-based crc32 function added. This is about 3 times faster than the slicing-by-8 implementation used previously. On gcc you can use -msse4.2 -mpclmul to enable it.
SSE4.2 and AVX2 optimized progressive encoding on JPEGs. This is about 20% faster.
Undocumented arithmetic coding support with "--arithmetic" added.
Several bugs fixed.

For us end users, is there compiled version for windows 64bit?

[Malloc Voidstar]

v0.3 compiled for Windows with msys2 GCC 5.3.0. "AVX2" builds are actually -march=native -mtune=native on Haswell-E, AFAIK any AVX2-capable CPU should be able to run these.

[dado023]

v0.3 compiled for Windows with msys2 GCC 5.3.0. "AVX2" builds are actually -march=native -mtune=native on Haswell-E, AFAIK any AVX2-capable CPU should be able to run these.

Thank you Malloc

[nikkho]

I released version 0.3.
Changes:
As always, more speed and compression for PNG.
PCLMULQDQ-based crc32 function added. This is about 3 times faster than the slicing-by-8 implementation used previously. On gcc you can use -msse4.2 -mpclmul to enable it.
SSE4.2 and AVX2 optimized progressive encoding on JPEGs. This is about 20% faster.
Undocumented arithmetic coding support with "--arithmetic" added.
Several bugs fixed.

Would you please post 0.3 Win32 and Win64 binaries in the releases page?

[lorents17]

fhanau
Whether you could let out pngwolf-zopfli with your zopfli version?

[AiZ]

Hello,

fhanau, could you quickly explain differences between presets please ?

                 (e7200@2.86GHz)
enwik8      size     time
-2      35354277     0m29
-3      35140806     0m34
-4      34978611     0m40
-5      35014164     0m50
-6      34991292     1m25
-7      34987354     1m56
-8      34984430     6m10
-9      34984430     6m10

I remember -M1 being the same than -M2 for gzip/png in previous version of ECT.

Thanks,

AiZ

[a902cd23]

I just compressed some PNG with ADVpng and PNGout and picked the smallest, then i ran ECT 0.3 -9 --allfilters on all files, saved a few % more.

All original files and all recompressed files was then converted to BMP with faststone viewer.

Some files had this difference
00000026: 23 46
00000027: 2E 5C
0000002A: 23 46
0000002B: 2E 5C

I think it is DPI 300 vs 600. Some original is 600 and after recompressed 300. Is there a limit of 300 dpi in ECT?

These 4 bytes is the only difference on just some files.

EDIT: My misstake - as usual - it´s ADVpng and PNGout that does this, ECT keeps 600 dpi.

[lorents17]

del

[fhanau]

v0.4 is out.
In the summer, I will further improve compression and try to get more of this merged into zopfli.

[Malloc Voidstar]

Win32/Win64 binaries, with multi-threading and folder support

[a902cd23]

To Malloc Voidstar

I have i7-3770k with win7x64 and i cannot run ECT_x64.exe, program crash. ECT_x86.exe does work. Yes, 3770 does not have AVX2, only AVX!

After extracting all three exe to ramdisk and copied newly taken photo (P1140977.JPG), commandline was:
for %a in (ect*.exe) do ( copy P1140977.JPG %a.jpg & %a -9 %a.jpg )

Tried -1 to.

[Bulat Ziganshin]

PCLMULQDQ-based crc32 function added. This is about 3 times faster than the slicing-by-8 implementation used previously.

i can't believe! slicing-by-16 has the speeds up to 4 GB/s (i.e. 1 cycle/byte), but speed greatly depends on the compiler's ability to optimally arrange instructions. what's your speed with sse implementation?

[Malloc Voidstar]

To Malloc Voidstar

I have i7-3770k with win7x64 and i cannot run ECT_x64.exe, program crash. ECT_x86.exe does work. Yes, 3770 does not have AVX2, only AVX!

After extracting all three exe to ramdisk and copied newly taken photo (P1140977.JPG), commandline was:
for %a in (ect*.exe) do ( copy P1140977.JPG %a.jpg & %a -9 %a.jpg )

Tried -1 to.

Does this build work?