defluff - a deflate huffman optimizer

[roytam1]

*bump*
any news of defluff?

[nikkho]

*bump*
any news of defluff?

Same question here...

[porneL]

*bump* again. I was unable to contact author via sourceforge e-mail. I'd love to see defluff released with a license that would allow me to reuse and redistribute it.

[roytam1]

Joachim had you time to improve Defluff?
I've started some preliminary work to update the zlib, I'd like to patch it to directly produce optimized Huffman trees (the same way Deflopt and Defluff do) a lot of apps could benefit from this. I know that Deflopt and Defluff do some tricks that go beyond simple Huffman trees optimizations (like replacing some LZ matches of length 3 by 3 literals) but it would still be a nice first step to get the smallest possible Huffman headers right from zlib.

What I've done so far was to fix the way the lib computes the size of the blocks, it now keeps the length in bits for maximum precision. Next I will start to dump various Huffman headers to compare those produced by the lib and Deflopt/Defluff.

If you disclose your code using a license compatible with zlib I could reuse your work and your name would of course be credited in the patch. This would help me a lot (and avoid me a few headache).

and news about this patch?

[caveman]

and news about this patch?

I had not much time working on it, but I've submited a small hack back in April to the zlib mailing list that saves 13 bytes on enwik8.

http://mail.madler.net/pipermail/zli...il/003047.html

This optimization is also done by defluff but not by deflopt.

zlib trees.c replacement attached here.

[lorents17]

Found a bug in the optimization of the image. As I understand the problem in Photoshop ICC profile.

[m_ueberall]

Could it be that defluff always discards the png pHYs chunk (cf. specification) specifying the physical pixel dimensions ("grep pHYs........." works for the input files, but not for the output generated by defluff)? In that case, is it possible to implement this as well as all existing tools that easily let you set/re-add this information seem to insist on also re-encoding the image data which of course defeats the purpose of using defluff in order to ensure that the output file is as small as possible?

[caveman]

Could it be that defluff always discards the png pHYs chunk

It does not:

$ pngcheck -v out2.png
File: out2.png (66107 bytes)
  chunk IHDR at offset 0x0000c, length 13
    256 x 256 image, 24-bit RGB, non-interlaced
  chunk gAMA at offset 0x00025, length 4: 0.45455
  chunk pHYs at offset 0x00035, length 9: 2835x2835 pixels/meter (72 dpi)
  chunk tIME at offset 0x0004a, length 7:  2 Jan 2015 19:56:19 UTC
  chunk IDAT at offset 0x0005d, length 8192
    zlib: deflated, 32K window, maximum compression
  chunk IDAT at offset 0x02069, length 8192
  chunk IDAT at offset 0x04075, length 8192
  chunk IDAT at offset 0x06081, length 8192
  chunk IDAT at offset 0x0808d, length 8192
  chunk IDAT at offset 0x0a099, length 8192
  chunk IDAT at offset 0x0c0a5, length 8192
  chunk IDAT at offset 0x0e0b1, length 8192
  chunk IDAT at offset 0x100bd, length 362
  chunk IEND at offset 0x10233, length 0
No errors detected in out2.png (14 chunks, 66.4% compression).

$ defluff <out2.png >out3.png
defluff 0.3.2
Copyright (c) 2010-2011 Joachim Henke

finished successfully

$ pngcheck -v out3.pngFile: out3.png (65967 bytes)
  chunk IHDR at offset 0x0000c, length 13
    256 x 256 image, 24-bit RGB, non-interlaced
  chunk gAMA at offset 0x00025, length 4: 0.45455
  chunk pHYs at offset 0x00035, length 9: 2835x2835 pixels/meter (72 dpi)
  chunk tIME at offset 0x0004a, length 7:  2 Jan 2015 19:56:19 UTC
  chunk IDAT at offset 0x0005d, length 65854
    zlib: deflated, 32K window, maximum compression
  chunk IEND at offset 0x101a7, length 0
No errors detected in out3.png (6 chunks, 66.4% compression).

To my knowledge it merges IDAT chunks, does various optimizations in Deflate compressed chunks (removes some matches of length 3, Huffman header tweaking...) but leaves uncompressed chunks untouched.
You can check if a pHYs chunk is present using pngcheck as shown here.

[m_ueberall]

It does not:
...
To my knowledge it merges IDAT chunks, does various optimizations in Deflate compressed chunks (removes some matches of length 3, Huffman header tweaking...) but leaves uncompressed chunks untouched.
You can check if a pHYs chunk is present using pngcheck as shown here.

I stand corrected, sorry for the noise.
Thanks a lot for the pointer to pngcheck--after looking at the script in question which actually invokes a number of different tools beforehand, the pHYs chunk was not removed by defluff, but by TruePNG (and only because I explicitly requested that using "-md remove all"). Last time I simply overlooked this preprocessing step--DOH.

Kind regards, Markus

[necros]

src code?

[drsassafras]

I was hoping that you would let this program be repackaged and distributed. Do you give consent for such to happen? I didn't find a licence associated with the software.

[nikkho]

I was hoping that you would let this program be repackaged and distributed. Do you give consent for such to happen? I didn't find a licence associated with the software.

Being it posted as attachments in the forum, that is implying redistribution is allowed, I guess.

[drsassafras]

Being it posted as attachments in the forum, that is implying redistribution is allowed, I guess.

Not good enough. It does not provide a legal framework for distribution. Copyright is still in the hands of the author, so distribution or even use of it would strictly speaking constitute a violation of copyright law. So if you used it you have broken the law. It matters more if redistribution starts to take place. (yes I know thats not what people do in an informal manner)

Ive compared the PNG compression of what defluff can do with what ImageOptim application can achieve. Defluff can compress images further, even after ImageOptim has done its best. I contacted ImageOptim and asked if they wanted to integrate defluff into the application. There answer was yes, but it would have to be released under an open source licence for them to do so.

[dado023]

would it get more compression after treating PNG with ECT ?

For example, compress PNG with ECT, then compress with Defluff.

[nikkho]

would it get more compression after treating PNG with ECT ?

For example, compress PNG with ECT, then compress with Defluff.

Indeed that is what FileOptimizer does.

[dado023]

Indeed that is what FileOptimizer does.

tnx Nikkho

OT: will you soon have free time to churn out new latest and greatest Fileoptimizer version?

[nikkho]

tnx Nikkho

OT: will you soon have free time to churn out new latest and greatest Fileoptimizer version?

There has been some behind the scenes work: https://sourceforge.net/p/nikkhokkho/code/HEAD/tree/
But not yet enought for a new release.

[SvenBent]

you should save the results from both aka

the one after ECT
and the one after ECT > efluff

and mix the two together with cavemans huffmix..

Huffmix is awesome when you are using diffent png optimization tools

[guindos98]

Seems that defluff (both 32 & 64 bit versions) deletes some of my files on these two zip files I've attached. I don't know why, and I don't know if that could happen with other zip files.