Suggestions for Preprocessors/filters or file specific compression.

[SvenBent]

Being new to this forum. please bear with me if i'm asking questions all ready asked and answered.

I want suggestions for compression preprocessor/filter or file specific compression programs.

so far I'm using:

7-zip - General compression / storage container
Winrar - General compression / storage container
CCM(x) - General compression

ECM - CD-image preprocessor
Packjpg - JPEG compression
SoundSlimmer - MP3 compression

JPEGoptim - Jpeg lossless optimizer
PNGout - PNG lossless optimizer
MP3 repacker - MP3 lossless optimizer


I tried using XML-wrt but it never did any improvements on single file test

is there anything else i could throw into the mix.

I tried lpeg and winuda, but they where to slow for my likings

[Surfer]

StuffIt - JPEG lossless

[SvenBent]

I'm using PackJPG for that.
its freeware and CLI witch make it easy to make batch files for.

[SvenBent]

I'm just experimentation with this method and it seem to work

BMP fiels coudl be compressed with BMFc.
then decoded again. to a new bmp.

then you use xdelta 3.0q to make a patch file

the bmf file + the diff can the restore the originale BMP bit indetical.

original.bmp -> .7z (ultra9 = 3.399bytes
.BMF + diff ->.7z (storage) = 2.858bytes

I'm trying to se what can be done with CCMx

[SvenBent]

compression the original.bmp with CCMx gave a size of 2.985 bytes

So BFMc +diff is stiller smaller.

if BMF is open source that may be i a good way to compress bmp data

[Black_Fox1]

How about trying UHARC or Libra8 on BMPs?

[SvenBent]

Doign some more test

WinRAR (with filters) 4.219 bytes
WinRar (no filters) 4.134 bytes
WinUHA 2.882bytes

bmf +diff is stille the samlles way

But the size of the decompresser is not taking into account so it will only work wit many or big bmps

[squxe]

Document Press 1.11

Program for compressing compound files. It can compress MS Office documents: Word, Excel, PowerPoint files and big number of other files. Document Press reduces file size by removing dead space. THERE IS NO ANY DATA LOSS. After compression you can use these files without any decompression.

http://www.7-zip.org/igor.html

P.S.
Where can I download win version of JPEGoptim?

[SvenBent]

i might upload it somewhere tomorrow (to busy today) seem its a bit hard to find a windows version today.

[squxe]

thanks Sven

[Black_Fox1]

Where can I download win version of JPEGoptim?

http://onlinezoologists.com/people/w....2.2_win32.zip

THINGS BROKEN UNDER WINDOWS
- no wildcards
- directory related functions broken

Some Perl scripts for little more comfortable usage can be downloaded here

[LovePimple]

Document Press 1.11
Program for compressing compound files. It can compress MS Office documents: Word, Excel, PowerPoint files and big number of other files. Document Press reduces file size by removing dead space. THERE IS NO ANY DATA LOSS. After compression you can use these files without any decompression.
http://www.7-zip.org/igor.html

A link to the latest version of DP (6.01a) was posted earlier.

http://www.encode.su/forums/index.php?action=vthre ad&forum=1&topic=506

[squxe]

thanks LovePimple!

[maadjordan]

btw the "document Press v6.01a" work on CHM by:

1- remove comment heading at begin of file.
2- increase the LZX compression from LZX-16 to LZX-17

so it works as chm-optimizer

[squxe]

so it works as chm-optimizer

Yea!
winrar.chm - 307720
after optimization - 279282 (9.2 % improved)
BlueSoleil.chm - 1510990
after optimization - 1465674 (3 % improved)
lzx16 -> lzx17
It works well!

[maadjordan]

if you need a real optimizer for CHM then it exist in my mind..

1- decompress CHM file
2- optimize HTML by removing comments and other non-viewable ref.
3- optimize GIF files with Trout's GIF Optmizer.
4- optimize PNG files with optipng,advpng,advdef,pngout,deflopt
5- optimize jpeg files with "jpegoptim"
6- compess the result in a new CHM with LZX-21 (i have not tested viewing chm files with LZX-21 on windows because it i don't have one).

even more..can be done on stage (3)
if gif files are still type (one frame) then convert to PNG with optipng and replace its name.gif with name.png in all html files..(this one is best in reducing CHM file..)

[squxe]

6- compess the result in a new CHM with LZX-21

How can I do it?
All programs that I know use only LZX17...

[maadjordan]

i don't know..how unless someone made it or igor provide an updated version of DocumentPress with LZX-21

[SvenBent]

About using BMF +diff as bmp compression it seem i might just be lucky with my first test file.

i tested some other bmp file but most og them was bit identical after BMF compression/decompression. so i could not use them as test files .

when i fimally found one that was different after the BMF compression/decompression it was better to just compress it with 7-zip

[Black_Fox1]

LZX-21

CABARC?

[Fallon]

Just reading this, it may not be usable here, but lzx-21 is used by 'makecab.exe' of course.
Recently I dumped winace (no longer developed as far as I know) and looked for a simple shell just for makecab.exe.
Cabpack looked easy (note: it renames makecab.exe to makecab.ovl).
I would have packed it up with some other relevant info, but am too busy in real life. So will drop it here.

All released cabpack versions 1998-1999 (1.4a was last update of 1.4):
http://www.sharemation.com/~fallon/cabpack_version s_12-13-14-14a_1998-1999.7z

Later makecab.exe versions are limited to specific windows versions.
Most recent 'makecab.exe' that still works on all windows versions, could be
makecab.exe_v5.1.2600.2172_(mpedp)_2005-06-17.exe

Included in:
http://www.sharemation.com/~fallon/mediaplayer_ent erprise_employment_pack_mpedp_10.00.00.3802.7z

Find more online, probably except for the older cabpack versions.

[maadjordan]

yes i'm talking about making CHM with LZX level 21 instead of 16 or 17.. no tool exist even the libmspack not support it Yet

[maadjordan]

there is another way to reduce a CHM file .. and i have tested it and its great on general cases.

a- tools required

1- "key tools" from http://www.keyworks.net/keytools.htm (download and install)
2- "PureHELP Compiler" from http://freenet-homepage.de/gnozal/PureHELPC.zip (unpack to its folder)
3- "MScompiler" from http://freenet-homepage.de/gnozal/MSHelpCompiler.rar (unpack to "PureHELP Compiler" folder)

b- the process
1- "decompile" with "Key Tools" .. this tool will generate (*.HHP) project file that can mimic the original project file (and i think its about 100% ok as the author of "key tools" was a the project leader in microsoft of MS-HTML (which is CHM)) and make a folder for the unpacked CHM
2- then optimize using procedures mentioned in one of my replies:

here is a copy:
================================================== ===
if you need a real optimizer for CHM then it exist in my mind..

1- decompress CHM file
2- optimize HTML by removing comments and other non-viewable ref.
3- optimize GIF files with Trout's GIF Optmizer.
4- optimize PNG files with optipng,advpng,advdef,pngout,deflopt
5- optimize jpeg files with "jpegoptim"
6- compess the result in a new CHM with LZX-21 (i have not tested viewing chm files with LZX-21 on windows because it i don't have one).

even more..can be done on stage (3)
if gif files are still type (one frame) then convert to PNG with optipng and replace its name.gif with name.png in all html files..(this one is best in reducing CHM file..)
================================================== ===

3- then compile using "PureHELP Compiler" by
- set the compiler in "options" into the HHC.EXE from "MSHelpCompiler.rar"
- then choose the HHP file in the unpacked CHM folder.
- then press compile

4- then you have two choices:
1- to "recompile" with "keytools" into a new CHM file (to check it increases or decreases the files..) then use "document process v6.01"
2- or the vice-versa
both choices should give you same result..( a smaller CHM file)

and thats it..
-note: if the resulted file from (keytools) is the smallest and you use it.. then you can not unpack it with 7-zip (i'll contact IGOR about it..).. it gives "unsupported compression method.."

here is an example:

windows.chm from XP-home-sp2 (690,641 bytes)
1- "document press" only (411,102 bytes)
2- "PureHELP Compiler" as there nothing to optimize.. (250,703 bytes)
3- after "document press" again (224,304 bytes)
4- then "keytools" reompile.. (191,566 bytes)

or if 4 is before 3..
4- then "keytools" reompile.. (191,566 bytes)
3- after "document press" again (191,566 bytes)

so in this file "doument press" done nothing,, but this is one of the cases as someitmes "keytools" increase size not decrease..

I hope this info is good..

[maadjordan]

here is an old beta version of keytools v2.0 beta 1

http://nybumbum.ytu.ru/utilities/keytls20b1.zip

[Vacon]

Hello everyone,

How can I do it?
All programs that I know use only LZX17...

Does http://www.helesic.cz/en/helecabcompresor.php do the trick? Iirc it does...

Best regards!

[maadjordan]

i know about it but you cannot assign a cab compression directly as chm is files container like compound file with streams compressed with LZX-16

cab = container with diffrent compression modes (none, mszip, quantum, lzx)

chm = is compressed html with images with certain linkers (in short) for full description look here

http://www.speakeasy.org/~russotto/chm/chmformat.h tml

[Fallon]

In addition to maad's reply: Helecab does not perform with optimal cab compression.
And compared to wellknown archivers, in a lot of cases the best result is achieved by just using 'makecab.exe' (cabarc should give the same result). Cabpack does that.

[maadjordan]

If you need best CAB maker then i should refer to MScab.exe http://mateusz.free.fr/mscab/ as it does the job and even more..

MsCab extends original (Microsoft) CAB format:
1. It allows you to add dirs' names to an archive.
2. You can encrypt files with password.
3. You can pack more than 65535 files per cabinet.
You can make standard CAB archives -- by not using these three extensions.

and it has a much advanced SFX files.. you can use in GUI with "total commander" & "FAR"

and here another SFX maker which work as GUI of MSCAB
http://www.imach.uran.ru/creatsfx/

[Fallon]

Maad, you sure are willing to walk the extra mile for some more bytes!
Making optimized cabs like this could lead to problems during installation of windows. There can be a check for unsupported cabs and setup will halt on finding some. Cabinet archives remain very much a Microsoft thing.
For msi, again it's makecab, even with problems like short file names. Using 'cabinet files' for everyday archiving is probably not a good idea either (if only because other formats compress better now).
So.... I just want to be able to make regular cabs on occasion.
Two links back at you. Microsofts components info:
http://www.mdgx.com/add.htm#DEV
http://www.programchecker.com/file/2356.aspx

[maadjordan]

my approach is to encourge developing lzx,mszip,quantum preprocessor (that if used in archivers like paq or others would reduce compressed file like what precomp did..). and to provide a CHM optimizer (i think it has commercial profit..btw my idea is not compyrighted and it will never be..)

if you know Netopsystem FEAD which was used in "adobe acrobat reader" it produce a installation of 33,7MB (unpacked) while it was 19,9MB (Netopsystem) and if you compressed the unpacked files with any program you know .. you'll never reach original size.. but if you unpack everthing (msi,cab files..) then you can reach.. and even beet it.. so the preprocessor for CAB file is already done..but comprecially .. while for example "precomp" has accomplished more by having zip,zlib flavors,gif,swf,tiff and i hope cab to be considered as cab-mscab compression is a variant of phil katz "zip" format so its rather easier to implement even LZX is easy but mszip compression is weak in relation to LZX.. so a much improved compression is worthit for mszip rather LZX...

and my taking about these mscab that exceed the cab format limit is not to be deployed with any windows installation as i prefer 7-zip.. to me its more practical..