Huffmix: a PNGOUT -r catalyst

[caveman]

Hello,
let me introduce you a new tool called Huffmix.

As you probably know PNG and Gzip compression is based on Deflate.
A Deflate stream is made of one or more blocks.
PNGOUT/KZIP lets you choose how many blocks it produces (-n option, or -b to set the block split threshold).
PNGOUT/KZIP also has a random switch (-r randomized initial tables "good for many trials with same options", this is -rn in KZIP).
PNGSlim alike scripts used to run hundreds of random trials in a row and only kept the smallest PNG file produced (this takes a lot of time).

Huffmix compares two files produced in random mode and cherry-picks the smallest blocks to combine them into a new file.

For instance if we have 3 blocks.
File A:
block 1: 110 bits
block 2: 317 bits
block 3: 272 bits
Overall: 699 bits

File B:
block 1: 104 bits
block 2: 312 bits
block 3: 289 bits
Overall: 705 bits

Until now optimization scripts would have kept File A and deleted File B (since A is smaller than B), Huffmix combines File A and File B into File C by picking the smallest blocks, this gives:

File C:
block 1: 104 bits (from File B)
block 2: 312 bits (from File B)
block 3: 272 bits (from File A)
Overall: 688 bits

We end up with a smaller file and managed to take profit of the calculations done to produce the biggest file, it works like a catalyst.
Of course since there's randomness involved it will not always work as demonstrated, nevertheless my first tests suggests that it actually works pretty well.

After about 20 cycles of pngout + deflopt + defluff + huffmix these 3 already optimized files are now a few bytes smaller:
bigmac.png 33906 -> 33902 bytes
getadrink.png 65317 -> 65293 bytes
mouse.png 105035 -> 104866 bytes


Huffmix 0.6b1 is available for testing.

Usage: huffmix [options] file1.[png|gz] file2.[png|gz] output.[png|gz]
Options:
-v Verbose
-q Quiet
-k Keep same block sequence as in first file

output can be the same as file1 or file2 (this is a fast and easy way to screw your file if there's a bug in Huffmix, you have been warned)

Version 0.6b2
- new option force
- about 20% faster

Version 0.6b1
- solves some "Could not find matching sub-blocks" problems

Version 0.5b4
- internal number of Deflate blocks limit set to 1024
- a memory allocation bug correction (led to rare segfaults or unexpected EOF)

Version 0.5b2/b3
- handles .gz files (gzip files created by kzip2gz)
- internal number of Deflate blocks limit set to 600
- a memory allocation bug correction (led to rare segfaults)

Version 0.4b2
- new -k and -q options
- Huffmix has its own webpage : http://frdx.free.fr/huffmix/

Version 0.4b1
- no longer copies the input file if it's the same as the output.
- allows you to combine files with different numbers of blocks (e.g. -n3 and -n5).

 File Type C-Offset C-Length U-Offset U-Length
   A    2         0   289296        0   200724
   A    2    289296    11622    31014    46169
   A    2    300918    34023    3c46d    54457

 File Type C-Offset C-Length U-Offset U-Length
   B    2         0   110543        0    75987
   B    2    110543   104305    128d3    67393
   B    2    214848    75383    23014    57344
   B    2    290231    11608    31014    46169
   B    2    301839    34029    3c46d    54457

 File C-Offset C-Length
   A         0   289296
   B    290231    11608
   A    300918    34023

IDAT new size 41872 (0xa390) bytes, saved 14 bits, output file size 41929 bytes

Occasionally this will produce files that have different blocks compared to those produced by PNGOUT, use the -k option to preserve the block sequence.


To ensure that PNGOUT will not modify filter or palette entries it has to be called with these options:
pngout -ks -kp -f6

There was a bug in PNGOUT in -f5 mode. Use the latest version of PNGOUT (May 2012 or February 2013) with Huffmix.

For instance a simple Huffmix roundtrip looks like this:

pngout -force -ks -kp -f6 -r -y my_file.png my_temp_file.png
defluff <my_temp_file.png >my_defluffed_temp_file.png
huffmix my_file.png my_temp_file.png my_file.png
huffmix my_file.png my_defluffed_temp_file.png my_file.png

or

kzip -rn -y my_temp_file.zip my_file
kzip2gz my_temp_file.zip my_temp_file.gz
defluff <my_temp_file.gz >my_defluffed_temp_file.gz
huffmix my_file.gz my_temp_file.gz my_file.gz
huffmix my_file.gz my_defluffed_temp_file.gz my_file.gz

Start over again.

Todo list:
- FreeBSD version
- Code clean up
- Handle type zero blocks
- Add support for full .zip files
- Remove limitation on internal number of Deflate blocks (1024 since 0.5b4)

[lorents17]

The interesting idea, and is the version for Windows?

[m^2]

Todo list:
- Mac OS X port
- Windows port
- Smarter file copying when nothing to do
- Handle asymmetric blocks
- Handle type zero blocks

[zyzzle]

Results look good, and it is a very great application for those who want to squeeze the last bytes out of PNG files.

How easy would it be to port the Linux version to DOS?

[caveman]

How easy would it be to port the Linux version to DOS?

If by DOS you mean Windows NT/2000/XP/Vista/7 command line... it's available now.

[caveman]

As I wrote earlier I suspected a bug in PNGOUT when using the -f6 option, Ken Silverman was kind enough to check it out and he has found a bug in... -f5 mode.
A new version of PNGOUT is available today (dated 07/02/2011).

And before lorents17 shows up with smaller versions of the images I used, here is what I get after a few hours:
bigmac.png 33902 -> 33900 bytes (2 bytes smaller)
getadrink.png 65293 -> 65286 bytes (7 bytes smaller)
mouse.png 104866 -> 104784 bytes (82 bytes smaller)

[caveman]

pngdb has been updated and is now called defdb, it has its own thread here.

I needed a tool to quickly display some internals of the Deflate stream to compare files produced by Huffmix. I make it available here since it could be helpful to others.

For each Deflate block pngdb displays:
- its type: 0 uncompressed, 1 fixed Huffman tree, 2 dynamic Huffman tree
- its boundary: this is the offset in the uncompressed data where a block ends and a new one starts.
- its compressed size in bits: self-explanatory
and a short summary: full size of the stream in bits, number of blocks

It looks like this:

T Boundary     Size
2        0     3163
2      dcc    16476
2     2955    18865
2     42e2    18606
2     5b9a    18296
2     7427     8514
2     817b    14479
98399 bits long, composed of 7 blocks

It can also be called with two options:
-s : displays only the overall Deflate stream size in bits
-d : dumps the Deflate stream in human readable form:
- [size] XX, a literal value XX in hexa (ranges from 00 to FF that is 0 to 255), took size bits compressed.
- [size] (length, distance), LZ pair in decimal, length (3 to 25 distance (1 to 3276, took size bits compressed.
- [size] EoB, End of Block, took size bits compressed.
Huffman trees of type 2 blocks are not dumped (I may add this feature in the future), nor are type 0 blocks.
It looks like this:

 [3] 00
 [9] (3,1)
 [5] 1E
 [3] 00
[11] (7,1)
 [5] 21
[10] (8,9)
 [4] 40
 [4] 80
 [8] (6,9)
...
[12] (9,432)
[12] (9,450)
[12] (9,468)
[12] (9,486)
 [6] EoB

[caveman]

Version 0.4b1 released (first post updated).

[caveman]

Version 0.4b2 released (first post updated).

The new -k option allows you to keep the same block sequence as in the first file.

For instance you have a file with 3 blocks and an other one with 5 blocks, without the -k switch Huffmix could have produced an output file made of 3, 4 or 5 blocks, you can now enforce to keep the same block sequence as in the first file with the -k option.

For instance if we have 3 and 5 blocks.
File A:
block 1: 110 bits
block 2: 317 bits
block 3: 272 bits
Overall: 699 bits

PNGOUT -n5 option has split block 2 and 3 in two.
File B:
block 1: 104 bits
block 2: 201 bits (was part of block 2 in File A)
block 3: 114 bits (was part of block 2 in File A) blocks 2&3 315 bits
block 4: 148 bits (was part of block 3 in File A)
block 5: 141 bits (was part of block 3 in File A) blocks 4&5 289 bits
Overall: 708 bits

Huffmix without -k outputs this file:
File C:
block 1: 104 bits (from File B)
block 2: 201 bits (from File B since blocks 2&3 are smaller than block 2 from File A)
block 3: 114 bits (from File B same as above)
block 4: 272 bits (from File A since blocks 4&5 are bigger than block 3 from File A)
Overall: 691 bits

Huffmix with -k outputs this file:
File D:
block 1: 104 bits (from File B)
block 2: 317 bits (from File A to preserve block sequence)
block 3: 272 bits (from File A to preserve block sequence)
Overall: 693 bits

Jonathon Fowler has build the new version of PNGOUT for Linux, Mac OS X and FreeBSD.

Using this version produced different block boundaries on some files leading to smaller files:
bigmac.png 33900 -> 33864 bytes (36 bytes smaller)
getadrink.png 65286 -> 65225 bytes (61 bytes smaller)
mouse.png 104784 -> 104740 bytes (40 bytes smaller)

[caveman]

Added Linux x86 64-bits versions of Huffmix and Pngdb.

[Jaff]

error 404 not found: http://frdx.free.fr/huffmix/huffmix_...86_current.zip

[caveman]

error 404 not found: http://frdx.free.fr/huffmix/huffmix_...86_current.zip

You can download it from the first message over-here.
A new version supporting Gzip files is on its way.

[caveman]

Version 0.5b2 released (first post updated).

Version 0.5b2
- handles .gz files (gzip files created by kzip2gz)
- internal number of Deflate blocks limit set to 600
- a memory allocation bug correction (led to rare segfaults)

[Jaff]

Thank you for the update!

[caveman]

Windows version 0.5b2 has been replaced by 0.5b3 (to fix some unpexpected crash)

[lorents17]

Good afternoon!
Wanted to learn, whether zip support will be realized?

[caveman]

Sorry, full zip archive support has very low priority do not expect it this year, but some minor updates are in the works.

Version 0.5b4 was silently released (it corrected a bug submitted by Aaron Reitz).

[caveman]

Version 0.6b1 released (first post updated).
- solves some "Could not find matching sub-blocks" problems

[SvenBent]

would ti be possible to add multiple file support, so vi can find the best block from more files in one run would make it easier to batch.

[caveman]

would ti be possible to add multiple file support, so vi can find the best block from more files in one run would make it easier to batch.

Had another similar request... I'm a bit bit busy right now but I'll try to add this feature.

[SvenBent]

I'm finally updating my png optimizing batch and implanting huffmix and i really love this utility.

please tell me if im wrong in this.
it would be possible to run multi instance of pngout /r and huffmix trials in parralel and then just huff mix the resulting files in the end
e.g i run 4 batch files tha does 200 iterations of png /r and huffmix individually. and then the 4 resulting files i huffmix together. i would end up with the optimal files of block from all 800 pngout /r trials.

or wold it be more efficient to run just one run of 800 trials ? i believe the compression ratio should be identical (except random is random) but am i wrong.?

if it works huffmix makes it pretty easy to multi thread pngout /r trials this way

[caveman]

It should work, there's only one problem that could occur:
The randomness is apparently based on time, if you fire two (or more) pngout instances exactly at the same time they may use the same random seed and produce exactly the same file. Now this has probably been mitigated by version 20120530 (and newer) of pngout, since on the Unix ports page it is stated : "Better randomness when the -r switch is used.", eventually ask Ken about it.

Beware that palette ordering may change during a standard pngout -r run and since this produces an entirely new image huffmix will not work, you have to pick a palette first (truepng or pngout itself) and then lock it using -kp when doing the random runs.

Don't forget to run deflopt and defluff (I run deflopt/defluff/deflopt again) right after pngout before huffmix.

I'm finally updating my png optimizing batch and implanting huffmix and i really love this utility.

please tell me if im wrong in this.
it would be possible to run multi instance of pngout /r and huffmix trials in parralel and then just huff mix the resulting files in the end
e.g i run 4 batch files tha does 200 iterations of png /r and huffmix individually. and then the 4 resulting files i huffmix together. i would end up with the optimal files of block from all 800 pngout /r trials.

or wold it be more efficient to run just one run of 800 trials ? i believe the compression ratio should be identical (except random is random) but am i wrong.?

if it works huffmix makes it pretty easy to multi thread pngout /r trials this way

[SvenBent]

Wait deflopt before huffmix ? i used it after my pngout /r and huffmix trials

Im actually integrating huff mix more and more into every stage possible. Even on trials on different block sizes cause its fast and just might give a few bytes more.
it is fun seeing how a bigger files outpout from pngout still results in a smaller file in the end after huffmixing it

 In:  908975 bytes               magicolor01.grey.png.thread1.png /c0 /f5 /d8
Out:  909590 bytes               magicolor01.grey.png.tmp1.png /c0 /f6 /d8
Chg:    +615 bytes (100% of original)
IDAT new size 908787 (0xdddf3) bytes, saved 1047 bits, output file size 908844 b
ytes

[SvenBent]

i just did some trials of using deflopt in the png out /r mixing or just waiting and do deflopt at the end.
Im using the kodak testsuit but missing pictures 8 (I messed it up)
however this is just one interation of pngout /r + pngout of the original. i copied the /r output file and made an extra deflopted version of it. so all in all i had 3 starting files

Original
/r
/r deflopted

Then i mixed "original" + "/r" into mixedNO
Then i mixed "original" + "/r deflopted" into mixdefl

i wrote down the sizes

Then i deflopted all the mixedno ("original" + "/r") and wrote their size as "deflopt at end"
I deflopted all the mixedelf ("original" + "/r deflopt") again and wrote down their size as "Deflopted mixed +end"

funny part is here the one that is only deflopted at the end is actually smaller in average than the ones deflopted both inside the mixing and at the end.
Lastly i just mixed the "deflopt end" + "Deflopt mixed +end"


the sizes :

MIXED NO DEFLOPT         14,252,970 bytes

MIXED WITH DEFLOPT       14,252,719 bytes
DEFLOPT AT END           14,252,443 bytes
DEFLOPT MIXED+END        14,252,456 bytes

Mixed of END/mixed+end   14,252,441 bytes


MIXED NO DEFLOPT
04/25/2014  20:12           694,608 kodim01.png.mixedno.png
04/25/2014  20:12           607,858 kodim02.png.mixedno.png
04/25/2014  20:12           482,799 kodim03.png.mixedno.png
04/25/2014  20:12           621,624 kodim04.png.mixedno.png
04/25/2014  20:12           758,955 kodim05.png.mixedno.png
04/25/2014  20:12           604,207 kodim06.png.mixedno.png
04/25/2014  20:12           556,755 kodim07.png.mixedno.png
04/25/2014  20:12           561,963 kodim09.png.mixedno.png
04/25/2014  20:12           569,433 kodim10.png.mixedno.png
04/25/2014  20:12           600,153 kodim11.png.mixedno.png
04/25/2014  20:12           546,449 kodim12.png.mixedno.png
04/25/2014  20:12           803,982 kodim13.png.mixedno.png
04/25/2014  20:12           694,534 kodim14.png.mixedno.png
04/25/2014  20:12           595,772 kodim15.png.mixedno.png
04/25/2014  20:12           532,702 kodim16.png.mixedno.png
04/25/2014  20:12           597,972 kodim17.png.mixedno.png
04/25/2014  20:12           754,566 kodim18.png.mixedno.png
04/25/2014  20:12           637,207 kodim19.png.mixedno.png
04/25/2014  20:12           475,903 kodim20.png.mixedno.png
04/25/2014  20:12           636,224 kodim21.png.mixedno.png
04/25/2014  20:12           689,341 kodim22.png.mixedno.png
04/25/2014  20:12           552,015 kodim23.png.mixedno.png
04/25/2014  20:12           677,948 kodim24.png.mixedno.png
MIXED NO DEFLOPT         14,252,970 bytes



MIXED WITH DEFLOPT
04/25/2014  20:14           694,591 kodim01.png.mixdelf.png
04/25/2014  20:14           607,845 kodim02.png.mixdelf.png
04/25/2014  20:14           482,799 kodim03.png.mixdelf.png
04/25/2014  20:14           621,615 kodim04.png.mixdelf.png
04/25/2014  20:14           758,952 kodim05.png.mixdelf.png
04/25/2014  20:14           604,199 kodim06.png.mixdelf.png
04/25/2014  20:14           556,743 kodim07.png.mixdelf.png
04/25/2014  20:14           561,936 kodim09.png.mixdelf.png
04/25/2014  20:14           569,422 kodim10.png.mixdelf.png
04/25/2014  20:14           600,148 kodim11.png.mixdelf.png
04/25/2014  20:14           546,437 kodim12.png.mixdelf.png
04/25/2014  20:14           803,968 kodim13.png.mixdelf.png
04/25/2014  20:14           694,526 kodim14.png.mixdelf.png
04/25/2014  20:14           595,752 kodim15.png.mixdelf.png
04/25/2014  20:14           532,688 kodim16.png.mixdelf.png
04/25/2014  20:14           597,955 kodim17.png.mixdelf.png
04/25/2014  20:14           754,566 kodim18.png.mixdelf.png
04/25/2014  20:14           637,180 kodim19.png.mixdelf.png
04/25/2014  20:14           475,891 kodim20.png.mixdelf.png
04/25/2014  20:14           636,212 kodim21.png.mixdelf.png
04/25/2014  20:14           689,336 kodim22.png.mixdelf.png
04/25/2014  20:14           552,011 kodim23.png.mixdelf.png
04/25/2014  20:14           677,947 kodim24.png.mixdelf.png
MIXED WITH DEFLOPT       14,252,719 bytes



DEFLOPT AT END
04/25/2014  20:19           694,583 kodim01.png.mixedno.png
04/25/2014  20:19           607,837 kodim02.png.mixedno.png
04/25/2014  20:19           482,787 kodim03.png.mixedno.png
04/25/2014  20:19           621,595 kodim04.png.mixedno.png
04/25/2014  20:19           758,941 kodim05.png.mixedno.png
04/25/2014  20:19           604,167 kodim06.png.mixedno.png
04/25/2014  20:19           556,736 kodim07.png.mixedno.png
04/25/2014  20:19           561,928 kodim09.png.mixedno.png
04/25/2014  20:19           569,401 kodim10.png.mixedno.png
04/25/2014  20:19           600,126 kodim11.png.mixedno.png
04/25/2014  20:19           546,435 kodim12.png.mixedno.png
04/25/2014  20:19           803,960 kodim13.png.mixedno.png
04/25/2014  20:19           694,514 kodim14.png.mixedno.png
04/25/2014  20:19           595,729 kodim15.png.mixedno.png
04/25/2014  20:19           532,686 kodim16.png.mixedno.png
04/25/2014  20:19           597,949 kodim17.png.mixedno.png
04/25/2014  20:19           754,537 kodim18.png.mixedno.png
04/25/2014  20:19           637,178 kodim19.png.mixedno.png
04/25/2014  20:19           475,886 kodim20.png.mixedno.png
04/25/2014  20:19           636,198 kodim21.png.mixedno.png
04/25/2014  20:19           689,323 kodim22.png.mixedno.png
04/25/2014  20:19           552,007 kodim23.png.mixedno.png
04/25/2014  20:19           677,940 kodim24.png.mixedno.png
DEFLOPT AT END           14,252,443 bytes



DEFLOPT MIXED+END
04/25/2014  20:24           694,583 kodim01.png.mixdelf.png
04/25/2014  20:24           607,837 kodim02.png.mixdelf.png
04/25/2014  20:24           482,787 kodim03.png.mixdelf.png
04/25/2014  20:24           621,595 kodim04.png.mixdelf.png
04/25/2014  20:24           758,942 kodim05.png.mixdelf.png
04/25/2014  20:24           604,170 kodim06.png.mixdelf.png
04/25/2014  20:24           556,739 kodim07.png.mixdelf.png
04/25/2014  20:24           561,928 kodim09.png.mixdelf.png
04/25/2014  20:24           569,401 kodim10.png.mixdelf.png
04/25/2014  20:24           600,126 kodim11.png.mixdelf.png
04/25/2014  20:24           546,436 kodim12.png.mixdelf.png
04/25/2014  20:24           803,958 kodim13.png.mixdelf.png
04/25/2014  20:24           694,514 kodim14.png.mixdelf.png
04/25/2014  20:24           595,735 kodim15.png.mixdelf.png
04/25/2014  20:24           532,686 kodim16.png.mixdelf.png
04/25/2014  20:24           597,949 kodim17.png.mixdelf.png
04/25/2014  20:24           754,537 kodim18.png.mixdelf.png
04/25/2014  20:24           637,178 kodim19.png.mixdelf.png
04/25/2014  20:24           475,887 kodim20.png.mixdelf.png
04/25/2014  20:24           636,198 kodim21.png.mixdelf.png
04/25/2014  20:24           689,323 kodim22.png.mixdelf.png
04/25/2014  20:24           552,007 kodim23.png.mixdelf.png
04/25/2014  20:24           677,940 kodim24.png.mixdelf.png
DEFLOPT MIXED+END        14,252,456 bytes



Mixed of END/mixed+end
04/25/2014  20:29           694,583 kodim01.png.end.png
04/25/2014  20:29           607,837 kodim02.png.end.png
04/25/2014  20:29           482,787 kodim03.png.end.png
04/25/2014  20:29           621,595 kodim04.png.end.png
04/25/2014  20:29           758,941 kodim05.png.end.png
04/25/2014  20:29           604,167 kodim06.png.end.png
04/25/2014  20:29           556,736 kodim07.png.end.png
04/25/2014  20:29           561,928 kodim09.png.end.png
04/25/2014  20:29           569,401 kodim10.png.end.png
04/25/2014  20:29           600,126 kodim11.png.end.png
04/25/2014  20:29           546,435 kodim12.png.end.png
04/25/2014  20:29           803,958 kodim13.png.end.png
04/25/2014  20:29           694,514 kodim14.png.end.png
04/25/2014  20:29           595,729 kodim15.png.end.png
04/25/2014  20:29           532,686 kodim16.png.end.png
04/25/2014  20:29           597,949 kodim17.png.end.png
04/25/2014  20:29           754,537 kodim18.png.end.png
04/25/2014  20:29           637,178 kodim19.png.end.png
04/25/2014  20:29           475,886 kodim20.png.end.png
04/25/2014  20:29           636,198 kodim21.png.end.png
04/25/2014  20:29           689,323 kodim22.png.end.png
04/25/2014  20:29           552,007 kodim23.png.end.png
04/25/2014  20:29           677,940 kodim24.png.end.png
Mixed of END/mixed+end   14,252,441 bytes

This results of this test is that it seems to be better to not use deflopt inside the mixing compared to just do it at the end. Unless you again remix everything.

Sadly im off to work. ima make a batch run over more files with several runs now that i found my method to compare it

[caveman]

Yes you should use deflopt before huffmix and you should even use option -b (saves the file even if could not save an entire byte but just a few bits).

Usually I do not rely on threshold sizes but number of blocks -n, since different thresholds could lead to the same number of blocks, thresholds may be interesting for large files.
When huffmixing files with different numbers of blocks always huffmix those holding more blocks first, for instance if you produced files holding between 2 and 8 blocks using the random option, huffmix the files holding 8 and 7 blocks first and then the one with 6 blocks and so on.

Wait deflopt before huffmix ? i used it after my pngout /r and huffmix trials

Im actually integrating huff mix more and more into every stage possible. Even on trials on different block sizes cause its fast and just might give a few bytes more.

[SvenBent]

yeah i just came back to report i didint use /b switch whcih would probably be better.

ima set up a batch run just to do some testing that cna run while im at work. ty for feedback

[SvenBent]

ok here are some testing of trucolor images (next up is gray tone... just because) of whatever to use deflopt at the end or in every iteration og pngout /r

Image Source: RGB8bit form http://www.imagecompression.info/test_images/
Method:
I first made 140 iterations of each file with pngout /r /f6 /force
sadly i discovered that my original was not mix compatible with my pngout iterations so i removed the original and recopied files form iteration 1 and used as "original'"

I then mixed original with iteration N into original,for the mixing with no deflopt results.
i did this in sets of 35 iterations and noted down results
The resulting files got save in a new MixedNO folder

I recopied files from iteration 1 into the spot for original files thereby having the same original starting files
I then did a new batch that deflopted iteration N. Then mixed Original with Iteration N into original files.
I again did this test in sets of 35 and noted down the results
The resulting files was copied into folder MixedWith.

In mixedNO and Mixed with folder i created a subfolder called DefloptedEND.
I copied the files from each folder into there respective DefloptedEND subfolder
ran deflopt on the files in both DefloptedEND folder

This proces was to insure that there was no different between the /r iterations from each test stride. as different between test strides could affect the and scew the result
The defloptedEND files of mixedWITH did no decrease in size and is therefore no in the info below
The reason i noted down after each 35 iterations was because i wanted to compared them in subsets but i forgot to deflopt the MixedNO files so i couldn't compared it in the end anyway

Deflopted 140Trials     227,965,129 bytes
Deflopted at the end    227,968,065 bytes


No Deflopt 35trials
04/29/2014  01:46         1,493,757 artificial.png
04/29/2014  01:50        62,439,821 big_building.png
04/29/2014  01:53        47,827,971 big_tree.png
04/29/2014  01:55        19,694,512 bridge.png
04/29/2014  01:56         9,311,583 cathedral.png
04/29/2014  01:57        21,153,221 deer.png
04/29/2014  01:58         5,398,142 fireworks.png
04/29/2014  01:58         2,989,063 flower_foveon.png
04/29/2014  01:58         6,823,395 hdr.png
04/29/2014  01:59        11,270,035 leaves_iso_1600.png
04/29/2014  02:00        10,139,929 leaves_iso_200.png
04/29/2014  02:00         7,418,650 nightshot_iso_100.png
04/29/2014  02:01        12,385,190 nightshot_iso_1600.png
04/29/2014  02:02         9,840,290 spider_web.png
No Deflopt 35trials     228,185,559 bytes


No Deflopt 70trials
04/29/2014  02:43         1,493,640 artificial.png
04/29/2014  02:47        62,409,724 big_building.png
04/29/2014  02:50        47,827,407 big_tree.png
04/29/2014  02:51        19,680,829 bridge.png
04/29/2014  02:52         9,311,384 cathedral.png
04/29/2014  02:53        21,153,212 deer.png
04/29/2014  02:54         5,396,662 fireworks.png
04/29/2014  02:54         2,988,037 flower_foveon.png
04/29/2014  02:54         6,819,666 hdr.png
04/29/2014  02:55        11,257,206 leaves_iso_1600.png
04/29/2014  02:55        10,123,450 leaves_iso_200.png
04/29/2014  02:56         7,414,917 nightshot_iso_100.png
04/29/2014  02:57        12,385,148 nightshot_iso_1600.png
04/29/2014  02:57         9,830,673 spider_web.png
No Deflopt 70trials     228,091,955 bytes


No deflopt 105trials
04/29/2014  03:49         1,493,519 artificial.png
04/29/2014  03:53        62,405,606 big_building.png
04/29/2014  03:55        47,827,360 big_tree.png
04/29/2014  03:57        19,680,820 bridge.png
04/29/2014  03:58         9,311,138 cathedral.png
04/29/2014  03:59        21,153,204 deer.png
04/29/2014  04:00         5,394,256 fireworks.png
04/29/2014  04:00         2,986,561 flower_foveon.png
04/29/2014  04:00         6,818,073 hdr.png
04/29/2014  04:01        11,257,157 leaves_iso_1600.png
04/29/2014  04:02        10,122,894 leaves_iso_200.png
04/29/2014  04:02         7,412,697 nightshot_iso_100.png
04/29/2014  04:03        12,385,099 nightshot_iso_1600.png
04/29/2014  04:04         9,824,101 spider_web.png
No deflopt 105trials    228,072,485 bytes


No deflopt 140trials
04/29/2014  04:54         1,493,429 artificial.png
04/29/2014  04:57        62,324,746 big_building.png
04/29/2014  05:01        47,827,325 big_tree.png
04/29/2014  05:03        19,680,816 bridge.png
04/29/2014  05:04         9,311,060 cathedral.png
04/29/2014  05:04        21,153,203 deer.png
04/29/2014  05:05         5,393,544 fireworks.png
04/29/2014  05:06         2,985,768 flower_foveon.png
04/29/2014  05:06         6,813,635 hdr.png
04/29/2014  05:07        11,257,136 leaves_iso_1600.png
04/29/2014  05:07        10,111,691 leaves_iso_200.png
04/29/2014  05:08         7,408,903 nightshot_iso_100.png
04/29/2014  05:09        12,385,083 nightshot_iso_1600.png
04/29/2014  05:09         9,822,529 spider_web.png
No deflopt 140trials    227,968,868 bytes


Deflopted at the end
04/29/2014  16:00         1,493,387 artificial.png
04/29/2014  16:00        62,324,566 big_building.png
04/29/2014  16:00        47,827,324 big_tree.png
04/29/2014  16:00        19,680,795 bridge.png
04/29/2014  16:00         9,311,052 cathedral.png
04/29/2014  16:00        21,153,202 deer.png
04/29/2014  16:00         5,393,384 fireworks.png
04/29/2014  16:00         2,985,690 flower_foveon.png
04/29/2014  16:00         6,813,582 hdr.png
04/29/2014  05:07        11,257,136 leaves_iso_1600.png
04/29/2014  16:00        10,111,687 leaves_iso_200.png
04/29/2014  16:00         7,408,841 nightshot_iso_100.png
04/29/2014  16:00        12,385,027 nightshot_iso_1600.png
04/29/2014  16:00         9,822,392 spider_web.png
Deflopted at the end    227,968,065 bytes


---  ---  ---  ---  ---


Deflopted 35Trials
04/29/2014  05:54         1,493,713 artificial.png
04/29/2014  05:59        62,439,610 big_building.png
04/29/2014  06:03        47,827,485 big_tree.png
04/29/2014  06:05        19,694,485 bridge.png
04/29/2014  06:06         9,311,430 cathedral.png
04/29/2014  06:08        21,153,220 deer.png
04/29/2014  06:09         5,397,968 fireworks.png
04/29/2014  06:09         2,989,029 flower_foveon.png
04/29/2014  06:09         6,823,290 hdr.png
04/29/2014  06:11        11,270,035 leaves_iso_1600.png
04/29/2014  06:11        10,139,796 leaves_iso_200.png
04/29/2014  06:12         7,418,617 nightshot_iso_100.png
04/29/2014  06:13        12,385,118 nightshot_iso_1600.png
04/29/2014  06:14         9,840,180 spider_web.png
Deflopted 35Trials      228,183,976 bytes


Deflopted 70Trials
04/29/2014  12:37         1,493,595 artificial.png
04/29/2014  12:42        62,409,505 big_building.png
04/29/2014  12:46        47,827,288 big_tree.png
04/29/2014  12:48        19,680,807 bridge.png
04/29/2014  12:49         9,311,243 cathedral.png
04/29/2014  12:50        21,153,211 deer.png
04/29/2014  12:52         5,396,508 fireworks.png
04/29/2014  12:52         2,988,011 flower_foveon.png
04/29/2014  12:53         6,819,188 hdr.png
04/29/2014  12:54        11,257,206 leaves_iso_1600.png
04/29/2014  12:55        10,123,446 leaves_iso_200.png
04/29/2014  12:56         7,414,882 nightshot_iso_100.png
04/29/2014  12:57        12,385,082 nightshot_iso_1600.png
04/29/2014  12:58         9,830,574 spider_web.png
Deflopted 35Trials      228,090,546 bytes


Deflopted 105Trials
04/29/2014  13:01         1,493,473 artificial.png
04/29/2014  13:06        62,402,734 big_building.png
04/29/2014  13:10        47,827,242 big_tree.png
04/29/2014  13:12        19,680,798 bridge.png
04/29/2014  13:13         9,311,105 cathedral.png
04/29/2014  13:15        21,153,203 deer.png
04/29/2014  13:16         5,394,124 fireworks.png
04/29/2014  13:16         2,986,532 flower_foveon.png
04/29/2014  13:17         6,817,959 hdr.png
04/29/2014  13:18        11,257,157 leaves_iso_1600.png
04/29/2014  13:18        10,122,890 leaves_iso_200.png
04/29/2014  13:19         7,412,657 nightshot_iso_100.png
04/29/2014  13:20        12,385,033 nightshot_iso_1600.png
04/29/2014  13:21         9,823,992 spider_web.png
Deflopted 105Trials     228,068,899 bytes    


Deflopted 140Trials 
04/29/2014  15:35         1,493,386 artificial.png
04/29/2014  15:40        62,321,765 big_building.png
04/29/2014  15:44        47,827,207 big_tree.png
04/29/2014  15:46        19,680,795 bridge.png
04/29/2014  15:47         9,311,047 cathedral.png
04/29/2014  15:49        21,153,202 deer.png
04/29/2014  15:50         5,393,384 fireworks.png
04/29/2014  15:50         2,985,690 flower_foveon.png
04/29/2014  15:51         6,813,580 hdr.png
04/29/2014  15:51        11,257,136 leaves_iso_1600.png
04/29/2014  15:53        10,111,687 leaves_iso_200.png
04/29/2014  15:54         7,408,841 nightshot_iso_100.png
04/29/2014  15:55        12,385,018 nightshot_iso_1600.png
04/29/2014  15:55         9,822,391 spider_web.png
Deflopted 140Trials     227,965,129 bytes

Im about to relook into the files and try find the individuelle smallest files from each iteration to compared how much Huffmix increased the efficiency of doing pngout/r trials.
I just need to find an easy way to copy the smallest files from withing 140 sub folders

[chornobyl]

If files have same names, totalcommander can do the task.

[SvenBent]

TY but i figure out i didn need the file just the filesize so a copule of simple dirs and idid it manually. im just to lazy atm to get back to my main rig and write the results

[SvenBent]

NO MIXING
04/29/2014  15:35         1,495,944 artificial.140.png
04/29/2014  15:39        62,691,953 big_building.129.png
04/29/2014  15:42        47,830,297 big_tree.118.png
04/29/2014  12:46        19,711,142 bridge.39.png
04/29/2014  15:47         9,330,023 cathedral.138.png
04/29/2014  06:07        21,153,359 deer.11.png
04/29/2014  06:08         5,414,884 fireworks.30.png
04/29/2014  15:50         3,000,477 flower_foveon.107.png
04/29/2014  13:17         6,839,930 hdr.105.png
04/29/2014  13:17        11,272,024 leaves_iso_1600.89.png
04/29/2014  15:53        10,159,599 leaves_iso_200.130.png
04/29/2014  13:19         7,447,499 nightshot_iso_100.99.png
04/29/2014  15:54        12,385,325 nightshot_iso_1600.108.png
04/29/2014  13:21         9,897,267 spider_web.96.png
                         228,629,723 bytes

Deflopted 140Trials 227,965,129 bytes
Deflopted at the end 227,968,065 bytes
NO Huffmix 228,629,723 bytes

adding in huffmix in 140 trials of pngout /r gave an adition reduction of around 645kb. with very little time increase.

im doing a new test with 8 bit gray tones.
however this time ima look at time efficiency as well