zlib_jo

[Jan Ondrus]

This is just zlib recompression i did for paq8px taken out into separate tool. It is without recursion and temp file with one little speedup trick: for files with multiple streams -> only previously successful parameters (on previous streams) are tried for all blocks if at least one of them is successfull on first block.

To transform: zlib_jo e input output
To inverse transform zlib_jo d input output

[Gonzalo]

Win32, please?

[schnaader]

Comparison with Precomp v0.4.4 and AntiZ 0.1.4-alpha. "-t-fjmb" used for Precomp (disables GIF, JPG, Base64, bZip2) for a fair arrangement. Test system: Intel i5 M520 @ 2.4 GHz, 4 GB RAM

FlashMX.pdf (Source)
--------------------
Original: 4,526,946 bytes
zlib_jo: 26,959,536 bytes, encode: 1.551 s, decode: 0.777 s, verification OK
Precomp -cn -t-fjmb: 26,953,437 bytes, encode: 2.621 s, decode: 0.78 s, verification OK
AntiZ --notest: 26,971,853 bytes, encode: 16.981 s, decode: 0.808 s, verification OK

archlinux-2009.08-core-i686.iso (Source)
----------------------------------------
Original: 360,212,480 bytes
zlib_jo: 635,055,514 bytes, encode: 134 s, decode: 78 s, verification OK
  zlib_jo: (first "recursion") 635,241,372 bytes, encode: 94 s, decode: 60 s, verification OK
Precomp -cn -t-fjmb -intense -d0: (no recursion) 641,703,263 bytes, encode: 223 s, decode: 80 s, verification OK
Precomp -cn -t-fjmb -intense: (full recursion) 669,805,031 bytes, encode: 697 s, decode: 95 s, verification OK
AntiZ --notest: 635,167,894 bytes, encode: 83 s, decode: 69 s, verification OK
  AntiZ --notest: 635,354,222 bytes (first "recursion"): bytes, encode: 26 s, decode: 7.5 s, verification OK

silesia.zip (Source)
------------------------------
Original: 67,633,896 bytes
zlib_jo: 75,899,609 bytes, encode: 17 s, decode: 9 s, verification OK
Precomp -cn -t-fjmb: 88,129,962 bytes, encode: 37 s, decode: 5 s, verification OK
AntiZ --notest: 67,633,924 bytes, encode: 3 s, decode: 1s, verification OK
reflate v0c3 (unpacking + .hif): 211,938,580 bytes .unp (decompressed streams), 2,316 bytes .hif (zlib stream differences for reconstruction)

• Precomp's intense mode (also recompresses "raw" zLib streams) is standard for zlib_jo and AntiZ - and without temporary files, it's 2-3 times faster
• The ArchLinux image seems to show that both zlib_jo and AntiZ don't support recursion (Precomp's recursion depth is 2 here if not using -d0)
• Neither zlib_jo nor AntiZ get a result as good as Precomp (regarding size) when simulating recursion by calling them again. The first "recursion" only gives a small change, the second one doesn't do anything for both. But this could be some file format advantage in Precomp (like PNG) or partial matches (see below). Also, AntiZ's is much faster than zlib_jo in the "recursion"
• Silesia.zip is a hard one, as ZIP containers with large files are in general. Most of the streams won't recompress bit-to-bit identical, so e.g. zlib_jo decompresses only "dickens" and "sao" - easy way to see that is the "paq8px_v71 -0" output that says "Transform fails" whenever no full match was found
• Best tool for silesia.zip is reflate, it completely decompresses everything and there is very little overhead for reconstruction. It doesn't support recursion either, it's built-in compression leads to a compressed size of 47,668,557 bytes
• Precomp wins on silesia.zip because it supports partial matches, e.g. it decompresses 6,541,517 of 6,627,202 bytes from "reymont"
• I thought that AntiZ would support partial matches, too, perhaps I missed some CLI options. It also surprises me that AntiZ doesn't decompress anything here, I'd have expected at least "dickens" and "sao" like for zlib_jo
• Some output or a verbose mode for zlib_jo would be nice to see how many streams were detected/decompressed/recompressed, something like AntiZ's output ("zlib headers found", "valid zlib streams", "recompressed")
• No clue why AntiZ is so slow encoding FlashMX.pdf

[Bilawal]

Hey, it can find zlib header but can't find deflate I think because it processed a file that contain zlib streams but didn't processed a file with deflate streams i.e no header.