ZopfliPNG

[Lode]

We launched an alpha version of ZopfliPNG at http://code.google.com/p/zopfli/

I'll try to incorporate your feedback in the next version.

[caveman]

Great, I'll have a look, thanks for sharing.

--filters=[types]: filter strategies to try:
0-4: give all scanlines PNG filter type 0-4
m: minimum sum
e: entropy
p: predefined (keep from input, this likely overlaps another strategy)
b: brute force (experimental)
By default, if this argument is not given, one that is most likely the best for this image is chosen by trying faster compression with each type.
If this argument is used, all given filter types are tried with slow compression and the best result retained. A good set of filters to try is --filters=0me.

- Have you considered reusing the two heuristics described in PNGWolf?
I'm not talking about the whole genetic algorithm, just "number of distinct bytes" and "number of distinct bigrams" since in Bjorn Hohrmanns study they are mentioned as being easy to implement, run fast, and beat the default filter selector, or is "e: entropy" already related to those?
Could you describe in two lines how "b: brute force (experimental)" is working?

- Apparently there's no way to change the colortype of the images (Photoshop for instance does not produce real grayscale images: colortypes 0 and 4) and sometimes images with a small palette end up smaller when saved in truecolor (due to the PLTE chunk overhead), another relatively common optimization trick when dealing with images with 16 or less colors (bit depth < is do avoid the "pixel packing" (2.3. Image layout in RFC 2083) step, each byte only contains a single reference to the palette even if it could hold 2, 4 or 8 (pixel packing can lead to larger Huffman tables and behaves poorly when filtering).
Colorype 6 (RGBA) to colortype 4 (RGB) + tRNS reduction when there's only one fully transparent color would also be welcome (the same goes for colortype 4 to colortype 0+tRNS).

[Lode]

There is no way to specify the output colortype because the program chooses the optimal one itself. It counts the amount of colors and uses a palette of appropriate bit size if <= 256. For small images, it will automatically try both with and without palette, because the program is aware that sometimes the palette overhead is bigger than using true color. If all pixels are grey, it automatically chooses greyscale. If all pixels happen to have values fitting in 1, 2 or 4 bit grey types, it chooses that. It can also choose to use palette transparency, color keying, or true alpha channel depending on how many and what transparent colors there are.

The less things you need to specify manually, the handier to use. It is also a bit tricky to allow the user to enforce a color model because that may lead to information loss (e.g. specifying a greyscale type on a color image). Still, being able to specify it is a nice idea, thank you.

b: brute force is experimental because I have found only very few images where it gives the smallest output (and I would have left it out entirely if these few cases weren't there). It works worse than expected even though the idea seemed nice. It LZ77 compresses the scanline with each of the 5 filtertypes, and uses the filtertype where it was the smallest. Of course, since it does not take the previous data window or huffman into account, it is completely disconnected from the compression reality, but it usually even performs worse than simple heuristics.

[Bloax]

Hmm, any chance for some Win32 binaries?

As a sidenote, it seems like Javascript has to be enabled for linebreaks to work. That's somewhat unusual for a forum. o_O

[roytam1]

Hmm, any chance for some Win32 binaries?

As a sidenote, it seems like Javascript has to be enabled for linebreaks to work. That's somewhat unusual for a forum. o_O

compiled with perlmingw gcc 4.6.3, with -O3 -flto -s -static-libgcc (I hate gcc that it does not make linking libgcc library as static by default)

[SvenBent]

It counts the amount of colors and uses a palette of appropriate bit size if <= 256.

that very often not optimal
sometimes even though the colors is so loe you can use a smaller palleter its better to still keep a 8bits pallete because it makes each pixels byte aligned and therefor fits perfect with the filterings that is also byte aligned

nevertheless i'm running a lot of test of pngwolf with zopfli compression and pngout so this tool just made it to come into it

however im working on my laptop so it going a bit slow as im using some big sets (18hours to run my usual pngout all filter/tresshold optimizng bat O.o

[Bloax]

compiled with perlmingw gcc 4.6.3, with -O3 -flto -s

Thanks, but

[roytam1]

Thanks, but

reuploaded.

[Bloax]

Thanks again!
It also doesn't link to libSTDc++ by default either, however.
now go -static-libstdc++ adn kill teh DMNS wth GNUS

But yeah, that does sound quite silly for default behavior. (Almost as silly as wiping out all unidentified PNG chunks, and that's a biggie.)

because of course everyone is going to have the exact same libraries as where the compilation was performed
right guys

right


guys



question mark

[Bulat Ziganshin]

Bloax, i use -static which should link all the libs, and gcc 4.7.2 donaloaded from https://sourceforge.net/projects/min...urce=directory

[roytam1]

with -static-libstdc++ the result binary is huge.
VC2005 /Ox /GL build is uploaded.

[Bloax]

It may be big (huge is more >1MB), but it does work.
tanks

And well, I can't into compilation. Unless we count the mental breakdown I had last time I tried. :v

[caveman]

Quick test on 8 files (in the sample_x.zip archive), compared to CryoPNG 0.74 (PNGOUT+defluff+cryopng+huffmix):


zopflipng --iterations=500 --splitting=3 --filters=01234mepb --lossy_8bit --lossy_transparent

(smallest in bold) zopflipng / CryoPNG
sample_1.png 84 / 84
sample_2.png 3608 / 3455
sample_3.png 8865 / 8999
sample_4.png 9087 / 9152
sample_5.png 15167 / 14955
sample_6.png 49794 / 49680
sample_7.png 77280 / 78443
sample_8.png 232197 / 232737

[caveman]

I'll try to incorporate your feedback in the next version.

Could it be possible to check if the predefined filter is the same as another known filter strategy to avoid recomputing exactly the same file?
There's a lot of software based on libpng that will basically output PNGs using 01234m filters.

[Bulat Ziganshin]

for SREP, gcc-compiled binary is largest, but fastest too. although it greatly depends on optimization options, compiler versions and microoptimizations for specific compilers

[roytam1]

for SREP, gcc-compiled binary is largest, but fastest too. although it greatly depends on optimization options, compiler versions and microoptimizations for specific compilers

It is a different story about with and without stdc++ library involving .
without stdc++, size difference between gcc and msvc build is small.
with stdc++, gcc build is bigger with several hundred KBs.

[caveman]

sample_2.png 3608 / 3455
sample_5.png 15167 / 14955

Now this is really interesting, these two images can benefit from "dirty alpha cleaning" that's what cryopng does and apparently zopflipng does something equivalent with the --lossy_transparent parameter.
But it looks like cryopng has still a small advantage here, since if I take the file produced by CryoPNG and recompress it with zopflipng without --lossy_transparent (to keep the work done by cryopng) it gets smaller!

CryoPNG first followed by zopflipng --iterations=500 --splitting=3 --filters=01234mepb --lossy_8bit
sample_2.png 3454
sample_5.png 14952

I'll do more tests and provide more visual results soon, and I'll look at zopflipng code since it would be great if I could fit what cryopng does directly in-there.

[caveman]

I ran more tests with PNG-32 files to compare cryopng dirty alpha cleaning with ZopfliPNG --lossy_transparent.

So far it looks like ZopfliPNG has only a single way to replace RGB components of fully transparent pixels: it fills them with the last color encountered, it does this from the left to the right and it wraps around when it reaches the end of a line. Whereas cryopng really reverses the filtering to get a lot of zeros in the compressed data, this is apparently more efficient (a never published cryopng take2 did things a bit differently, with Cedric Louvrier we found that it could even be interesting to propagate color from bottom to top).


zopflipng --iterations=500 --splitting=3 --filters=01234mepb --lossy_8bit --lossy_transparent
CryoPNG 0.74 (level 6: uses cryopng, PNGOUT, defluff and huffmix)

(smallest in bold) ZopfliPNG / CryoPNG
cryo_sample_1.png 5825 / 5715
cryo_sample_2.png 5990 / 5917
cryo_sample_3.png 6830 / 6802
cryo_sample_4.png 13446 / 13527
cryo_sample_5.png 14942 / 14504
cryo_sample_6.png 14840 / 14648
cryo_sample_7.png 38373 / 38509
cryo_sample_8.png 50498 / 49997


CryoPNG first followed by zopflipng --iterations=500 --splitting=3 --filters=01234mepb --lossy_8bit

cryo_sample_1.png 5712 (best result)
cryo_sample_2.png 5920 (better than zopflipng alone, 3 bytes off best result, DeflOpt can still save 9 bytes)
cryo_sample_3.png 6589 (best result)
cryo_sample_4.png 13449 (worse than zopflipng alone, 3 bytes off best result, Deflopt can still save 5 bytes)
cryo_sample_5.png 14498 (best result)
cryo_sample_6.png 14674 (better than zopflipng alone, 26 bytes off best result, Deflopt can still save 2 bytes)
cryo_sample_7.png 38139 (best result)
cryo_sample_8.png 49704 (best result)

cryopngtk2 -f4 followed by zopflipng --iterations=500 --splitting=3 --filters=01234mepb --lossy_8bit brings cryo_sample_4.png down to 13367 bytes (best result).

It seems that ZopfliPNG is often better than PNGOUT (at least without random trials and huffmix) and that cryopng does a better job than -lossy_transparent, but cryopng uses a different strategy for each filter type and it's usually worthwhile to try mixed mode filtering afterwards which dramatically increases processing time.


Now let's peek behind the curtain!

This is how fully transparent pixels are handled by respectively:
cryopng -f2 / zopflipng --lossy_transparent / cryopng -f1

The "thermal view" at the bottom shows the compression efficiency.

The file presented here is cryo_sample_5.png
14498 bytes cryopng -f2 + zopflipng
14942 bytes zopflipng -lossy_transparent
14562 bytes cryopng -f1 + zopflipng

Notice the bright colors in the first column of ZopfliPNG thermal view, this is where it looses some bytes compared to cryopng -f1.
Since the first byte of each raw in a PNG file is in fact the filter type [0-4] there's no continuity between the last pixels of the previous raw and the first ones in the new raw, thus ZopfliPNG needs to "set the color" again and this comes at a cost.

All the files (minus cryo_sample_5.png cryopng -f1) are in the zip archive attached to this message.

I've started to merge cryopng and zopflipng, early result is available in the cryopng discussion.

[cssignet]

Few suggestions:

Palette optimizer (1): ZopfliPNG should be able to detect if one value in the palette is transparent, then move it to the top; this should save some bytes by deleting the unuseful "255" value on some colors.

Palette optimizer (2): It can try to sort the colors in the palette, then using a different delta filtering than 0 (None) for Paletted images. Sometimes, it's more efficient to keep some unnecessary transparent values in tRNS (255) with a good palette organization.

Palette optimizer (3): If one entry is fully transparent in palette, it can be modified (RGB value) to optimize the sorting.

ColorType reduction: On few samples, grayscale images are better stored as Paletted, because of the precedent point (2), specially with gradients that have less than 200 colors, or those that contains some transparent values (GrayScale+Alpha > Paletted+tRNS).

Lossy transparency (1): Last year, i've tested the implementation used by ZopfliPNG and on a lot of cases it's worse than converting RGB data to 0,0,0. You should include trials, or using some combinations used by cryopng with different directions. It should be more efficient.

Lossy transparency (2): By using "--lossy_transparent" ZopfliPNG should assume that the output can be different than TrueColor+Alpha or GrayScale+Alpha. Once the program have deleted unnecessary RGB components, sometimes the output can be reduced to Paletted+tRNS.

​8 bits/pixels: It should be interesting to test 8 bits/pixels and 4 bits/pixels for images which have 16 colors or less. Sometimes 8 bits/pixels is better.

[caveman]

8 bits/pixels: It should be interesting to test 8 bits/pixels and 4 bits/pixels for images which have 16 colors or less. Sometimes 8 bits/pixels is better.

There's definitely a problem here, please check the 16 colors image in the archive, when "pixel packing" (each byte holds two 4-bits references to the palette) is activated the resulting file is bigger:

7533 bytes pal_16colors_4bits.png
7001 bytes pal_16colors_8bits.png

7539 bytes pal_16colors_zop.png
Obviously ZopfliPNG missed this optimization.



And for those interested in the "thermal view" pngthermal is now available.

[lorents17]

del

[Mr_KrzYch00]

64 bit static version of zopfliPNG just compiled with migw32 using x64 toolchain under windows 10 technical preview if anybody interested. Tested on windows 8.1 - working.
Also includes original zopfli.

Revision a29e46ba9f26
Sourcecode: https://github.com/google/zopfli

EDIT: Please check new thread for all the info and download links, thanks: http://encode.su/threads/2176-Zopfli...fliPNG-KrzYmod

[lorents17]

del