`pngwolf`

[Bjoern Hoehrmann]

Frederic Kayser asked me to post a pointer to 'pngwolf' http://bjoern.hoehrmann.de/pngwolf/ here. It is a PNG size reduction utility that tries to find good scanline filters and then compresses the filtered image data with 7-Zip's Deflate implementation. The source code is available at https://github.com/hoehrmann/pngwolf; build instructions are included there in the README; I don't provide binaries.

To find better scanline filters it employs a genetic algorithm and various heuristics, I wrote a bit about the technical details on the homepage and on the libpng mailing list, for instance http://article.gmane.org/gmane.comp....png.devel/4426.

It would help a lot if there was a Deflate implementation that can compress multiple versions of a string that vary only slightly very quickly. Right now 'pngwolf' uses zlib at a low level to recompresses all of the image data even if only one scanline filter is changed or if two scanline filter selections are merged (with a two-point crossover operation at the moment), which is very very wasteful; though even at relatively "fast" settings it does well when compared to other tools.

(My apologies to Frederic for misspelling his name, the forum software does not appear to support spelling it properly.)

[lorents17]

And it is possible to receive the compiled version?

[Intrinsic]

Wondered how long it'd take lorents to pop up ;p and btw lorents PNGKT screws up some images so you may want to remove it from your tool, see the CryoPNG thread here for a link to one of the images it messes up on. And i can't speak Russian so maybe you can report the problem to x128?

http://encode.su/threads/1260-CryoPN...t-introduction

[roytam1]

patch for making it compiles in mingw32/msys:

diff --git a/pngwolf.cxx b/pngwolf.cxx
index 5e94bcf..c3e76cf 100755
--- a/pngwolf.cxx
+++ b/pngwolf.cxx
@@ -18,37 +18,32 @@
 //
 ////////////////////////////////////////////////////////////////////
 
-#ifdef _MSC_VER
-#include <WinSock2.h>
-#pragma comment(lib, "ws2_32.lib")
-#else
-#include <arpa/inet.h>
+#ifdef __cplusplus
+extern "C" {
 #endif
 
-#include "Common/MyWindows.h"
-#include "Common/MyInitGuid.h"
-#include "7zip/IStream.h"
-#include "7zip/Compress/ZlibEncoder.h"
-#include "7zip/Common/FileStreams.h"
-#include "7zip/Common/InBuffer.h"
-#include "7zip/Common/StreamObjects.h"
-
+#define __STDC_LIMIT_MACROS
+#include <stdint.h>
 #include <signal.h>
 #include <stdlib.h>
-#include <stdint.h>
+#include <zlib.h>
+#include <float.h>
+#include <limits.h>
+#include <math.h>
+#include <stdio.h>
+
+#ifdef __cplusplus
+}
+#endif
+
 #include <ga/ga.h>
 #include <iostream>
 #include <list>
 #include <vector>
-#include <stdio.h>
 #include <ctime>
 #include <iomanip>
 #include <functional>
 #include <numeric>
-#include <zlib.h>
-#include <float.h>
-#include <limits.h>
-#include <math.h>
 #include <map>
 #include <bitset>
 
@@ -58,6 +53,25 @@
 #pragma warning(disable: 4100)
 #endif
 
+#ifdef _MSC_VER
+#include <WinSock2.h>
+#pragma comment(lib, "ws2_32.lib")
+#else
+#ifdef _WIN32
+#include <winsock2.h>
+#else
+#include <arpa/inet.h>
+#endif
+#endif
+
+#include "Common/MyWindows.h"
+#include "Common/MyInitGuid.h"
+#include "7zip/IStream.h"
+#include "7zip/Compress/ZlibEncoder.h"
+#include "7zip/Common/FileStreams.h"
+#include "7zip/Common/InBuffer.h"
+#include "7zip/Common/StreamObjects.h"
+
 ////////////////////////////////////////////////////////////////////
 // Miscellaneous structures and types
 ////////////////////////////////////////////////////////////////////
@@ -168,7 +182,7 @@ public:
   std::vector<char> original_unfiltered;
 
   // 
-  std::map<PngFilter, std::vector<char>> flt_singles;
+  std::map<PngFilter, std::vector<char> > flt_singles;
 
   //
   std::map<uint32_t, size_t> invis_colors;
@@ -559,7 +573,7 @@ void filter_idat(unsigned char* src, unsigned char* dst, const PngFilterGenome&
 ////////////////////////////////////////////////////////////////////
 // Signal handlers
 ////////////////////////////////////////////////////////////////////
-#ifdef _MSC_VER
+#if defined( _MSC_VER ) || defined( __MINGW32__ )
 BOOL WINAPI console_event_handler(DWORD Event) {
   switch (Event) {
   case CTRL_C_EVENT:
@@ -1704,7 +1718,7 @@ main(int argc, char *argv[]) {
 
   bool argOkay = true;;
 
-#ifndef _MSC_VER
+#if !defined( _MSC_VER ) && !defined( __MINGW32__ )
   sig_t old_handler;
 #endif
 
@@ -1920,7 +1934,7 @@ main(int argc, char *argv[]) {
 
   wolf.search_begun_at = time(NULL);
 
-#ifdef _MSC_VER
+#if defined( _MSC_VER ) || defined( __MINGW32__ )
   SetConsoleCtrlHandler(console_event_handler, TRUE);
 #else
   old_handler = signal(SIGINT, sigint_handler);
@@ -1934,7 +1948,7 @@ main(int argc, char *argv[]) {
   // not much to avoid that, other than setting a timer with
   // some grace perdiod which strikes me as too complicated.
 
-#ifdef _MSC_VER
+#if defined( _MSC_VER ) || defined( __MINGW32__ )
   SetConsoleCtrlHandler(console_event_handler, FALSE);
 #else
   signal(SIGINT, old_handler);

Notice: You need to edit CMakeCache.txt and add "-lws2_32" to CMAKE_CXX_STANDARD_LIBRARIES:STRING and CMAKE_C_STANDARD_LIBRARIES:STRING line manually to make it links.

MinGW (TDM-GCC) 4.5.2 stripped binary attached.

[lorents17]

roytam1 Thank you very much

[roytam1]

some benchmark:

pngoptone.bat:

@echo off

SET POOPFX=%~dp0

if NOT "%1"=="/b" GOTO PROC
SHIFT
SET EXECPFX=start /b /wait /belownormal

:PROC
 %POOPFX%binmay.exe -i "%~1" -o nul -s "00 00 00 08 61 63 54 4c" 2> nul
 if ERRORLEVEL 1 (
  %POOPFX%touch.exe -r "%~1" "%~1.time"
  %EXECPFX% %POOPFX%optipng.exe -zw32k -zc9 -zm9 -zs0,3 -f0,5 -i0 "%~1"
  if not ERRORLEVEL 1 (
   %EXECPFX% %POOPFX%advdef.exe -z -4 "%~1"
   %EXECPFX% %POOPFX%deflopt.exe -b "%~1"
   %EXECPFX% %POOPFX%pngout /y /b0 "%~1"
   if not ERRORLEVEL 2 (
    %EXECPFX% %POOPFX%advdef.exe -z -4 "%~1"
    %EXECPFX% %POOPFX%deflopt.exe -b "%~1"
   )
  )
  %POOPFX%touch.exe -r "%~1.time" "%~1"
  if exist "%~1.time" del "%~1.time"
 ) else (
  ren "%~1" "%~n1.apng"
 )

SET EXECPFX=
SET POOPFX=

pngwoptone.bat:

@echo off

SET POOPFX=%~dp0

if NOT "%1"=="/b" GOTO PROC
SHIFT
SET EXECPFX=start /b /wait /belownormal

:PROC
 %POOPFX%binmay.exe -i "%~1" -o nul -s "00 00 00 08 61 63 54 4c" 2> nul
 if ERRORLEVEL 1 (
  %POOPFX%touch.exe -r "%~1" "%~1.time"
  %EXECPFX% %POOPFX%optipng.exe -zw32k -zc9 -zm9 -zs0,3 -f0,5 -i0 "%~1"
  if not ERRORLEVEL 1 (
   %EXECPFX% %POOPFX%pngwolf.exe --in="%~1" --out="%~1"
   %EXECPFX% %POOPFX%deflopt.exe -b "%~1"
   %EXECPFX% %POOPFX%pngout /y /b0 "%~1"
   if not ERRORLEVEL 2 (
    %EXECPFX% %POOPFX%pngwolf.exe --in="%~1" --out="%~1"
    %EXECPFX% %POOPFX%deflopt.exe -b "%~1"
   )
  )
  %POOPFX%touch.exe -r "%~1.time" "%~1"
  if exist "%~1.time" del "%~1.time"
 ) else (
  ren "%~1" "%~n1.apng"
 )

SET EXECPFX=
SET POOPFX=

original png attached. xxxx_2.PNG and xxxx_3.PNG are copied from xxxx.PNG.

Log:

C:\Documents and Settings\Roy\Desktop>timer pngoptone.bat xxxx_2.PNG

Timer 3.01  Copyright (c) 2002-2003 Igor Pavlov  2003-07-10
OptiPNG 0.6.5: Advanced PNG optimizer.
Copyright (C) 2001-2011 Cosmin Truta.

** Processing: xxxx_2.PNG
748x391 pixels, 3x8 bits/pixel, RGB
Reducing image to 8 bits/pixel, 225 colors in palette
Input IDAT size = 28359 bytes
Input file size = 28489 bytes

Trying:
  zc = 9  zm = 9  zs = 0  f = 0         IDAT size = 6397

Selecting parameters:
  zc = 9  zm = 9  zs = 0  f = 0         IDAT size = 6397

Output IDAT size = 6397 bytes (21962 bytes decrease)
Output file size = 7214 bytes (21275 bytes = 74.68% decrease)

        7214        6284  87% xxxx_2.PNG
        7214        6284  87%

***                 DeflOpt V2.07                 ***
***       Built on Wed Sep  5 18:56:30 2007       ***
***  Copyright (C) 2003-2007 by Ben Jos Walbeehm  ***



"C:/Documents and Settings/Roy/Desktop/xxxx_2.PNG"
Number of bytes saved: 76 (6,284 --> 6,208)
File rewritten.


Number of files processed  :        1
Number of files rewritten  :        1
Total number of bytes saved:       76
19,752,516 cycles.
 In:    6208 bytes               xxxx_2.PNG /c3 /f0 /d8
Out:    6044 bytes               xxxx_2.PNG /c3 /f0 /d8, 225 colors
Chg:    -164 bytes ( 97% of original)
        6044        6044 100% xxxx_2.PNG (Bigger 6170)
        6044        6044 100%

***                 DeflOpt V2.07                 ***
***       Built on Wed Sep  5 18:56:30 2007       ***
***  Copyright (C) 2003-2007 by Ben Jos Walbeehm  ***



"C:/Documents and Settings/Roy/Desktop/xxxx_2.PNG"


Number of files processed  :        1
Number of files rewritten  :        0
Total number of bytes saved:        0
10,361,453 cycles.

Kernel Time  =     0.046 = 00:00:00.046 =   0%
User Time    =     0.015 = 00:00:00.015 =   0%
Process Time =     0.062 = 00:00:00.062 =   1%
Global Time  =     6.078 = 00:00:06.078 = 100%

C:\Documents and Settings\Roy\Desktop>timer pngwoptone.bat xxxx_3.PNG

Timer 3.01  Copyright (c) 2002-2003 Igor Pavlov  2003-07-10
OptiPNG 0.6.5: Advanced PNG optimizer.
Copyright (C) 2001-2011 Cosmin Truta.

** Processing: xxxx_3.PNG
748x391 pixels, 3x8 bits/pixel, RGB
Reducing image to 8 bits/pixel, 225 colors in palette
Input IDAT size = 28359 bytes
Input file size = 28489 bytes

Trying:
  zc = 9  zm = 9  zs = 0  f = 0         IDAT size = 6397

Selecting parameters:
  zc = 9  zm = 9  zs = 0  f = 0         IDAT size = 6397

Output IDAT size = 6397 bytes (21962 bytes decrease)
Output file size = 7214 bytes (21275 bytes = 74.68% decrease)

---
# 748 x 391 pixels at depth 8 (mode 3) with IDAT 292859 bytes (6397 deflated)
---
---
# 890 bytes smaller

***                 DeflOpt V2.07                 ***
***       Built on Wed Sep  5 18:56:30 2007       ***
***  Copyright (C) 2003-2007 by Ben Jos Walbeehm  ***



"C:/Documents and Settings/Roy/Desktop/xxxx_3.PNG"
Number of bytes saved: 75 (6,324 --> 6,249)
File rewritten.


Number of files processed  :        1
Number of files rewritten  :        1
Total number of bytes saved:       75
14,568,404 cycles.
 In:    6249 bytes               xxxx_3.PNG /c3 /f5 /d8
Out:    6044 bytes               xxxx_3.PNG /c3 /f0 /d8, 225 colors
Chg:    -205 bytes ( 96% of original)
---
# 748 x 391 pixels at depth 8 (mode 3) with IDAT 292859 bytes (5300 deflated)
---
- zlib deflated idat size:   11180 #   -17 bytes   +3 seconds
- zlib deflated idat size:   11150 #   -47 bytes   +3 seconds
- zlib deflated idat size:   11132 #   -65 bytes   +3 seconds
- zlib deflated idat size:   11131 #   -66 bytes   +4 seconds
- zlib deflated idat size:   11127 #   -70 bytes   +4 seconds
- zlib deflated idat size:   11114 #   -83 bytes   +4 seconds
- zlib deflated idat size:   11107 #   -90 bytes   +6 seconds
- zlib deflated idat size:   11099 #   -98 bytes   +6 seconds
- zlib deflated idat size:   11095 #  -102 bytes   +6 seconds
- zlib deflated idat size:   11061 #  -136 bytes   +6 seconds
- zlib deflated idat size:   11060 #  -137 bytes   +7 seconds
- zlib deflated idat size:   11049 #  -148 bytes   +8 seconds
- zlib deflated idat size:   11016 #  -181 bytes   +9 seconds
- zlib deflated idat size:   11004 #  -193 bytes   +9 seconds
- zlib deflated idat size:   10977 #  -220 bytes   +9 seconds
- zlib deflated idat size:   10969 #  -228 bytes   +9 seconds
- zlib deflated idat size:   10957 #  -240 bytes  +10 seconds
- zlib deflated idat size:   10955 #  -242 bytes  +10 seconds
- zlib deflated idat size:   10950 #  -247 bytes  +10 seconds
- zlib deflated idat size:   10948 #  -249 bytes  +11 seconds
- zlib deflated idat size:   10920 #  -277 bytes  +11 seconds
- zlib deflated idat size:   10910 #  -287 bytes  +12 seconds
- zlib deflated idat size:   10901 #  -296 bytes  +14 seconds
- zlib deflated idat size:   10892 #  -305 bytes  +16 seconds
- zlib deflated idat size:   10869 #  -328 bytes  +16 seconds
- zlib deflated idat size:   10845 #  -352 bytes  +17 seconds
- zlib deflated idat size:   10829 #  -368 bytes  +17 seconds
- zlib deflated idat size:   10813 #  -384 bytes  +18 seconds
- zlib deflated idat size:   10770 #  -427 bytes  +19 seconds
- zlib deflated idat size:   10760 #  -437 bytes  +21 seconds
- zlib deflated idat size:   10745 #  -452 bytes  +24 seconds
- zlib deflated idat size:   10720 #  -477 bytes  +25 seconds
- zlib deflated idat size:   10719 #  -478 bytes  +26 seconds
- zlib deflated idat size:   10717 #  -480 bytes  +27 seconds
- zlib deflated idat size:   10710 #  -487 bytes  +27 seconds
- zlib deflated idat size:   10699 #  -498 bytes  +29 seconds
- zlib deflated idat size:   10695 #  -502 bytes  +31 seconds
- zlib deflated idat size:   10689 #  -508 bytes  +32 seconds
- zlib deflated idat size:   10674 #  -523 bytes  +35 seconds
- zlib deflated idat size:   10672 #  -525 bytes  +37 seconds
- zlib deflated idat size:   10636 #  -561 bytes  +38 seconds
---
# 0 bytes smaller

***                 DeflOpt V2.07                 ***
***       Built on Wed Sep  5 18:56:30 2007       ***
***  Copyright (C) 2003-2007 by Ben Jos Walbeehm  ***



"C:/Documents and Settings/Roy/Desktop/xxxx_3.PNG"


Number of files processed  :        1
Number of files rewritten  :        0
Total number of bytes saved:        0
9,162,458 cycles.

Kernel Time  =     0.046 = 00:00:00.046 =   0%
User Time    =     0.015 = 00:00:00.015 =   0%
Process Time =     0.062 = 00:00:00.062 =   0%
Global Time  =    59.984 = 00:00:59.984 = 100%

[caveman]

some benchmark:

748x391 pixels, 3x8 bits/pixel, RGB
Reducing image to 8 bits/pixel, 225 colors in palette

You should try to run pngwolf on some real 24-bits images, this one has been reduced to a paletted version, choosing the right filters is more effective on 24-bits RGB or 32-bits RGBA images.

[roytam1]

You should try to run pngwolf on some real 24-bits images, this one has been reduced to a paletted version, choosing the right filters is more effective on 24-bits RGB or 32-bits RGBA images.

IMO identifying and providing suitable optimization for such image is one of main task of image optimizers.

[caveman]

Hello,
I did my own quick test run.

cubs.png (24-bits RVB 427x241) sizes in bytes:
187 640 optipng -o7
181 801 pngslim
181 164 pngwolf
180 971 pngwolf --max-stagnate-time=40 --7zip-mpass=15 --7zip-mmc=258 --7zip-mfb=258 --zlib-level=9

I also ran deflopt, defluff and deflopt again on each file.
Well done Bjoern!!

Smallest file enclosed.

[m^2]

Hello,
I did my own quick test run.

cubs.png (24-bits RVB 427x241) sizes in bytes:
187 640 optipng -o7
181 801 pngslim
181 164 pngwolf
180 971 pngwolf --max-stagnate-time=40 --7zip-mpass=15 --7zip-mmc=258 --7zip-mfb=258 --zlib-level=9

I also ran deflopt, defluff and deflopt again on each file.
Well done Bjoern!!

Smallest file enclosed.

On my test file your switches:
--max-stagnate-time=40 --7zip-mpass=15 --7zip-mmc=258 --7zip-mfb=258 --zlib-level=9
happened to be weaker than none at all. And it was so slow with them...

[Intrinsic]

I never understand why people try to run png over an jpeg image ;p plenty of appropriate images all over the web, like the awesome pixeljoint.


just a few examples.
http://www.pixeljoint.com/files/icons/full/ae.gif
http://www.pixeljoint.com/files/icon.../isocastle.png
http://www.pixeljoint.com/files/icon...ingsunrise.png
http://www.pixeljoint.com/files/icon..._the_woods.png

[caveman]

On my test file your switches:
--max-stagnate-time=40 --7zip-mpass=15 --7zip-mmc=258 --7zip-mfb=258 --zlib-level=9
happened to be weaker than none at all. And it was so slow with them...

I've also noticed that on this file bigmac.png (32-bits RVBA 128x12 sizes in bytes:
34670 original
34670 pngwolf --max-stagnate-time=40 --7zip-mpass=15 --7zip-mmc=258 --7zip-mfb=258 --zlib-level=9
34615 pngwolf
34535 pngwolf + (deflopt + defluff)x2
34501 original + deflopt + defluff
34481 optipng -o7
34480 optipng -o7 + deflopt + defluff
34057 pngslim
34055 pngslim + defluff
34022 cryogen-d
34021 cryogen-d + defluff
34021 cryogen-d + defluff + pngwolf
33997 cryogen-d + pngslim

Apparently pngwolf with the options is just unable to optimize further and the output file is the same as the original.

[lorents17]

cubs.png = 180 387

pngwolf --zlib-level=1 --zlib-strategy=0 --zlib-memlevel=9 --7zip-mpass=1
pngout -f6 -n5

Ken Silverman has updated PngOut under my request, the filter-f6 is added.

[lorents17]

bigmac = 33 943

cryopng -for -f1
pngout -f5 -n5

[willvarfar]

Ken Silverman has updated PngOut under my request, the filter-f6 is added.

Can you describe the idea in the -f6 filter?

[Intrinsic]

It keeps existing row filters and just uses it's enhanced deflate engine from what i can tell.

[caveman]

bigmac = 33 943

33 940 bytes, cryogen-x + defluff
33 938 bytes, cryogen-x + a lot of (pngout random trial + deflopt + defluff)

[caveman]

I never understand why people try to run png over an jpeg image ;p plenty of appropriate images all over the web, like the awesome pixeljoint.

These pictures are actually not appropriate to test pngwolf, filtering is way more effective on RGB and greyscale images compared to paletted ones. Paletted images usually take advantage of filter 0 (None).

Another sample getadrink.png (32-bits RGBA 200x294) sizes in bytes:
72023 original
71995 optipng -o7
70310 pngwolf
69857 pngwolf --max-stagnate-time=40 --7zip-mpass=15 --7zip-mmc=258 --7zip-mfb=258 --zlib-level=9
68340 cryopng -f0 + optipng -o7
66710 cryopng -f0 + pngwolf
66357 cryopng -f0 + pngwolf same options as above
65438 cryogen-d
65426 cryogen-x
65379 pngslim
65344 pngslim + a lot of (pngout random trial + deflopt + defluff)

[Intrinsic]

Yes i know they weren't it was just a general observation, plenty of other sources like deviantart etc for pngs.

[lorents17]

bigmac = 33 906
getadrink = 65 317

[caveman]

Pngwolf Linux x86 binaries (32-bits and 64-bits) enclosed.

[lorents17]

Good evening!
Whether somebody could alter pngwolf that it didn't use compression 7-zip, and used only zlib for compression PNG? Simply very much it is necessary.

And still, help to compile two versions advancecomp, I have created two versions advancecomp and it is impossible to compile.

[maadjordan]

Good evening!
Whether somebody could alter pngwolf that it didn't use compression 7-zip, and used only zlib for compression PNG? Simply very much it is necessary.

And still, help to compile two versions advancecomp, I have created two versions advancecomp and it is impossible to compile.

BTW I'v noticed you are using v1.15 .. try the latest one v1.16 from here http://advancemame.sourceforge.net/beta/

[lorents17]

To tell the truth, for me the version which I now am more important to compile have made, and to decide as to disconnect compression through 7-zip in pngwolf

[lorents17]

Good afternoon!
Help to change, please, pngwolf so that compression of the image passed through zlib, without use 7-zip. How it can be done?

Prompt how it is possible to compile pngwolf?

[AiZ]

Pngwolf linked against zlib 1.2.7 only.

pngwolf_zlib_only_win32.7z

Barely tested, be warned.

[lorents17]

AiZ Thank you very much

[lorents17]

Good evening!
Prompt, please, as it is possible to make so that pngwolf automatically touched values

 --zlib-level= --zlib-memlevel= --zlib-strategy=

i.e. that it was possible to specify

 --zlib-level=9 --zlib-memlevel=1-9 --zlib-strategy=0-3

and pngwolf automatically would touch all specified values?

[boogerlad]

Can someone provide the necessary patches to compile on linux and linked against zlib only?

EDIT: Actually, I've taken the matter into my own hands. https://github.com/boogerlad/pngwolf

[nikkho]

EDIT: Actually, I've taken the matter into my own hands. https://github.com/boogerlad/pngwolf[/QUOTE]
Would you post updated precompiled binaries?