FileOptimizer

[nikkho]

I have published a new file recompressor. In fact it is like in other similar tools, only a GUI to several command line utilities that do the optimization/recompression. Main difference here, is that it is not only supporting the usual PNG, and JPEG formats, but lots more. Interestingly in contains and updated Windows build of Gifsicle, and features a GZ and SWF recompressor of mine (I will talk about this later).

http://nikkhokkho.sourceforge.net/st...=FileOptimizer

Hope you enjoy it, and if you have any concerns, or suggestions, do not hesitate posting here.

[maadjordan]

- first Ghostcript plugin link or Full is not working for me. as i could not test pdf support.
- APNG support is not working try it on the APNg samples at wikipedia link http://en.wikipedia.org/wiki/APNG
- did you try PngOptimizer from www.pngoptimizer.org its good also for APNG and often can optimize png further.
- 3DM support not working. its deflate stream not zip container like apk and epub

[nikkho]

Hello maad, and thank you very much for your report.

As for Ghostcript plugin, the problem is my free web hoster does not allow to upload file bigger than 5 Mb, but in fact is as easy as copying gswin32c.exe and gsdll32.dll onto the plugins folder.

Relating the APNG support, make sure the extension of the file is .apng and not .png. The issue comes because some of the optizing tools I use, think it is a regular PNG, and remove the other frames. I will try to add such a detection regarless of the file extension for next FileOptimizer releases.

I was not aware of PngOptimizer, so I will take a look to it, and if something is gained, add it to the PNG chain in FileOptimizer.

Finally for 3DM, you are right, it is deflate and not ZIP, the same happened with SIS, so it is fixed now on the source code, and will be release with the next version.

[maadjordan]

still its a very welcome step into unifiing such tools under one GUI..

I've tested Ghostscript and its reduce by re-compressing images so please confirm what commands are used.

BTW you can use mp3packer with -z command to recompress losslessly mp3 http://omion.dyndns.org/mp3packer/mp3packer.html DL http://omion.dyndns.org/mp3packer/

[nikkho]

still its a very welcome step into unifiing such tools under one GUI..

I've tested Ghostscript and its reduce by re-compressing images so please confirm what commands are used.

BTW you can use mp3packer with -z command to recompress losslessly mp3 http://omion.dyndns.org/mp3packer/mp3packer.html DL http://omion.dyndns.org/mp3packer/

Thank you.

For PDF recompression I am using gswin32c -q -dBATCH -dNOPAUSE -dSAFER -dDELAYSAFER -dQUIET -dNOPROMPT -sDEVICE=pdfwrite -dPDFSETTINGS=/ebook -dCompatibilityLevel=1.5 -sOutputFile=output.pdf input.pdf. Theorically with the -dPDFSETTINGS=/ebook setting, it is lossy, since images greater than 150 dpi will be downsampled to 150 dpi, but it is unnoticeable, and the savings comparing with the default option, makes sense.

Thank you for pointing me out to MP3packer, I will integrate it. I guess that best switches should be -t -s -z

[nikkho]

Updated to version 1.40:
- Added MP3 recompression support.
- Improved ZIP compression ration upgrading to zRecompress 2.10.
- Improved APNG compression ratio by using PngOut and OptiPNG with them too.
- Improved PNG compression ratio including PngOptimizer.
- Improved PNG compression ratio by using tweaking OptiPNG parameters.
- Improved JPEG compression ratio and speed updating to jpegoptim 1.2.4 (custom built with libjpeg 8d).
- Assured that APNG files with PNG extension are properly detected regardless of the extension (maadjordan).
- Removed 3dm and sis from the ZIP chain since they are in fact deflate (maadjordan).
- Files grid column widths are now stored in the INI file in order to be permanent across executions.
- Some internal code simplification and refactorization.

[Stephan Busch]

What a nice program. Here are some thoughts:

*Is Deflopt already used as last optimizing step for PNG?
*would there be a way to let JPEGTRAN optimize JPEG inside PDF?
*does JPEGTRAN the same as EXIFIRON from photomolo package did?

impressive tool which I am using here on my private data.. thank you very much Javier

update: by default, JPEGTRAN does not keep EXIF tags so it is not lossless unless you
specify -copy all switch. I would also suggest to use -progressive instead of -optimize.

[nikkho]

What a nice program. Here are some thoughts:

*Is Deflopt already used as last optimizing step for PNG?
*would there be a way to let JPEGTRAN optimize JPEG inside PDF?
*does JPEGTRAN the same as EXIFIRON from photomolo package did?

impressive tool which I am using here on my private data.. thank you very much Javier

Glad you liked Stephan!
Yes, DeflOpt is used as the last step for PNG images (as well as the last step for GZ and ZIP).
I am not aware JPEGTRAN could optimize JPEG data inside PDF streams. Is that possible?
Yes JPEGTRAN is more or less similar, but apart of removing EXIF and other meta information, it optimizes Huffman encodings.

[Stephan Busch]

By default, JPEGTRAN does not keep EXIF tags so it is not lossless unless you
specify -copy all switch. I would also suggest to use -progressive instead of -optimize.

JPEGTRAN cannot look inside .PDF, so there must be some tool that gets the .JPEG from the .PDF, optimizes them with JPEGTRAN and paste them back.

[nikkho]

By default, JPEGTRAN does not keep EXIF tags so it is not lossless unless you
specify -copy all switch. I would also suggest to use -progressive instead of -optimize.

I was concerned of not using progressive JPEG, since it was not supported in some applications.

Fortunatelly, I have seen that now, including Flash 11.3 and later already support them, so it makes sense to convert them to progressive in order to squeeze its size a bit more. I will add it to the next FileOptimizer update.

Hope that arithmetic encoded JPEG images will get a broader support too, so I can switch to them.

[maadjordan]

i forget to mention that you can get v1.16 beta of advancecomp from here http://advancemame.sourceforge.net/beta/

regarding optimizing jpg streams in PDF:
- i would recommend having pure jpg stream (no exif or other streams) as no its for reading and not for photo editing.
- progressive is very welcome.
- you need to read jpeg stream optimize it then inject it back .. i think its difficult

[Stephan Busch]

yep, arithmetic encoded JPEG.. XnView does support them in latest version, but Paint.net, newest Windows Live Photogallery, Win8 and Office 2013 preview do still not support them. And the free code that does it was invented and open-source back in 2005..

[nikkho]

i forget to mention that you can get v1.16 beta of advancecomp from here http://advancemame.sourceforge.net/beta/

regarding optimizing jpg streams in PDF:
- i would recommend having pure jpg stream (no exif or other streams) as no its for reading and not for photo editing.
- progressive is very welcome.
- you need to read jpeg stream optimize it then inject it back .. i think its difficult

I did not know about AdvaceCOMP 1.16 beta, so I will update it for the next FileOptimizer release. Thank you again. Nevertheless, asuming the changelog is updated, I will not expect any recompression difference with it.

Will investigate a bit about strems in PDF, and let you know.

yep, arithmetic encoded JPEG.. XnView does support them in latest version, but Paint.net, newest Windows Live Photogallery, Win8 and Office 2013 preview do still not support them. And the free code that does it was invented and open-source back in 2005..

Indeed it is a real pitty application are not updated to support arithmetic JPEG. Specially if we consider that most of them use libjpeg or libjpeg-turbo, and adding support, is just a matter of a couple of defines.

[Stephan Busch]

Yep I agree. Other than newest software, you can also buy brand new camera models which also don't support progressive or arithmetic JPEG..

You may wish to add OGG REHUFF which losslessly optimizes .OGG files.

[nikkho]

You may wish to add OGG REHUFF which losslessly optimizes .OGG files.

It seems interesting... Can you please point me on where to get latest Windows binaries?

[Stephan Busch]

You can compile the sources mentioned at http://wiki.xiph.org/Rehuff
The author does not publish executables.
My compile is from Shelwien: http://nishi.dreamhosters.com/v/ogg_rehuff.rar

[nikkho]

You can compile the sources mentioned at http://wiki.xiph.org/Rehuff
The author does not publish executables.
My compile is from Shelwien: http://nishi.dreamhosters.com/v/ogg_rehuff.rar

I have integrated your binaries on FileOptimizer, and will be available in the next version.
I guess that only rehuff_theora.exe is needed and that rehuff.exe is a less powerful version, isn't it?

[Stephan Busch]

Yes I think so.
I have been searching for a method that optimizes .MSI Installers, but there doesn't seem to be a way (except Setup File Squeezer).
It can somehow compress them better than all other tested lossless tools.

[nikkho]

I have been searching for a method that optimizes .MSI Installers, but there doesn't seem to be a way (except Setup File Squeezer).
It can somehow compress them better than all other tested lossless tools.

Do not hesitate updating me here, I you find a new commandline tool for recompressing files, no matter if they are MSI or other types, I will add it to FileOptimizer!

[m^2]

pngwolf is the king of png optimisation, much better than OptiPNG/AdvanceCOMP, just thrash them and take it instead. Seriously.
And you can run defluff as another postprocessor.

[Stephan Busch]

I have tried to search for a reasonable PDF recompressor, but there doesn't seem to be a single free lossless recompressor.
To p.ex. recompress the deflate parts with advdef and jpeg parts with jpegtran we have to build our own program.
The same goes with .msi - unpackers are around but no lossless recompressors.

In my tests pngwolf is much too slow and does not optimize better than current FileOptimizer toolset.

[Stephan Busch]

What tool currently recompresses .TIFF files in FileOptimizer? All tags are deleted and a 17.9 MB .TIFF/deflate can hardly be optimized to 6.8 MB losslessly - can somebody recheck that please?

In my tests the .TIFF/deflate content was recoded to lossy .JPEG and tags were discarded.
Wouldn't it be better to optimize deflate content?

[nikkho]

pngwolf is the king of png optimisation, much better than OptiPNG/AdvanceCOMP, just thrash them and take it instead. Seriously.
And you can run defluff as another postprocessor.

Very nice contribution. I will add defluff as the latest step after Deflopt. Regarding pngwolf, are ther any already build win32 binaries available?

[nikkho]

IIn my tests pngwolf is much too slow and does not optimize better than current FileOptimizer toolset.

What tool currently recompresses .TIFF files in FileOptimizer? All tags are deleted and a 17.9 MB .TIFF/deflate can hardly be optimized to 6.8 MB losslessly - can somebody recheck that please?

In my tests the .TIFF/deflate content was recoded to lossy .JPEG and tags were discarded.
Wouldn't it be better to optimize deflate content?

Since TIFF files can be either deflated or JPEGed or Zipped, FileOptimizer invokes first the JPEG tool chain (jpegoptim + jpegtran) and then advzip + deflopt

To me it sounds quite strange this behaviour, I mean, I have no reasonably explanation on how it was losy converted to JPEG. Will do some tests with TIFF, and let you know. In the mean time, would be able to attach a couple of problematic TIFF files?

Thanks.

[m^2]

In my tests pngwolf is much too slow and does not optimize better than current FileOptimizer toolset.

In my tests it's faster than OptiPNG -o7 that FO uses, IIRC much faster. But beware, its speed is highly variable, don't test on just 1 file.
When it comes to strength I seriously doubt it can keep up with pngwolf+deflopt+defluff not to say min(deflopt, deflopt+defluff, pngwolf --even-if-bigger+deflopt, pngwolf --even-if-bigger+deflopt+defluff, pngwolf --even-if-bigger+pngout /f6 /force+deflopt, pngwolf --even-if-bigger+pngout /f6 /force+deflopt+defluff) (that I use), but can't check because it says 'ordinal 344 can't be found on COMCTL32.dll' (XP x64) and I don't bother to run the pipeline myself manually.

[m^2]

Regarding pngwolf, are ther any already build win32 binaries available?

I don't know. I have some, but they might be old, I don't follow the project.

[Stephan Busch]

here is one .TIFF/deflate: http://www.squeezechart.com/P5200262.tif
here is my PNGWOLF compile : http://www.squeezechart.com/pngwolf.exe
and also pngrewrite which can sometimes help: http://www.squeezechart.com/pngrewrite.exe

[nikkho]

Good! I will add pngwolf and pngrewrite on the next FileOptimizer release. For your information, pngrewrite 1.4.0 is available at http://entropymine.com/jason/pngrewrite

I have tried with FileOptimizer 1.40, and provided TIFF file has not been optimized, meaning that resulting image is the same as the original one. Find attached screenshot.

Can you please confirm me you are using same FileOptimizer version, and paste a screenshot of the execution?

[Stephan Busch]

I cannot reproduce that error with any of my .Tiff files. But ADVZIP and ADVDEF cannot optimize .Tiff - there seem to be no tool that can optimize deflated .Tiff. I have asked Cosmin Truta and Andrea Mazzoleni if they could add support for deflated .Tiff in OptiPNG/ADVDEF.
During my experiments I discovered that .PNG which are already optimized with all tools, can sometimes be recompressed with ADVDEF.

If you want I could host the File Optimizer full package with Ghostscript on my site.

[nikkho]

Thank you. So probably the issue is solved then.

I will remove advzip and deflopt from the TIFF tool chain, and will add advdef in the PNG one.

About your hosting offer, would be perfect for me if you are able to provide me with an FTP account that allow me to easily update contents. Is that possible?