PIM 2.00 (alpha) is here!!!

[encode]

Okay, this' the biggest update of PIM ever!

What's new, highlights:
+ Added the brand new PIM2 archive format featuring Store/PPMd compression and CRC32 checking
+ Fixed a bug with ZIP archives crash
+ Improved ZIP-file handling - now you can view encrypted ZIP archives (encrypted files marked with "+" symbol)
+ Now PIM will skip existing files during extraction
+ Many code optimizations and improvements
+ Improved GUI, Vista styled menu icons

Enjoy!

Note that this is just an alpha release! Be careful, program can contain bugs. Do not use it for important data and after each compression operation check an archive integrity (Test command)!

Please, check it for:
+ Compatibility with PIM v1.x
+ ZIP file support, including with ZIP files produced with WinRAR (InfoZIP) and WInZIP 10/11 (BZip2/PPMd)

Post your comments, impressions and suggestions here!

Link:
pim200.zip (362 KB)

[Nania Francesco]

Seem a well done job, compliment, but I expected me that the compression I inserted LZPM!

[LovePimple]

Awesome! Thanks Ilia!

[nimdamsk]

Nice minimalistic interface. But i'd like to see big buttons line

[encode]

Could you make it so that the archive is not solid and files can be separately added and extracted? Especially for extracting only some files this would greatly reduce time needed!

...
By the way, PIM v2 uses non solid archives, and this can be very useful with slow CM compression (upcoming PIMPLE2). You can simply extract any file - no need to extracting all previous files. For example if you have a large archive and you need just one file - you can get it fast!

[encode]

What's next:
+ I think I will add the LZPM compression as the main compression of PIMv2. The unreleased LZPM 0.07 will be used (which includes E8/E9 transformer similar to QUAD). With tunable parameter like Max optimise lookhead (Longest optimised match) and fast decompression it's a good candidate.
+ PPMd will kept for good overall and high text compression.
+ WIll wait with PIMPLE2 adding - algorithm too young!
+ I will continue improving ZIP file support.
+ Step-by-step will add the RAR-archive support (at least browsing support - for fun)
+ Some small GUI improvements
Huge list...

[Nania Francesco]

FANTASTIC NOTICE !

[squxe]

fast decompression its a good candidate

Great!

[LovePimple]

I like your plans!

[encode]

Another variant is LZMA. But, it's slower at compression. However, it shows very good perormance on binary data - and combining LZMA/PPMd (auto algo switching)...

In coparison, LZPM is my own compression technology. It has a fast compression (mode 0) and fast decompression (much like LZMA). It can be used as a replacement for Deflate - thus I call it modern LZ. Actualy it uses ROLZ. Can I call it ROLZ in my archiver?

Post your ideas here!

[squxe]

combining LZMA/PPMd (auto algo switching)

It seems to me, it will be great if youll include both lzma and lzpm in Pim (with settings for choice method)

[LovePimple]

I like the idea of adding LZMA!

[Nania Francesco]

LZMA is very good Idea!

[encode]

+ Some small GUI changes (WinRAR like file list)
+ Fixed bug with an encrypted zip file which contains a directory entries
+ Added a nice feature:
If compressed size is longer than original - file will be just stored with no compression. Of course super small files (< 8 bytes) not compressed. (PPMd cannot compress a file less than 8 bytes long).

It's funny, but WinZip 11 has no such feature - compressed file can be larger than original.

[encode]

Got some problems adding LZPM (memory allocation inside a thread). Will delay LZPM adding. Instead, later, will try to add something different like LZMA/BZip2 or even Deflate.
And currently will keep compression as is, but keep improving GUI, features, and support for ZIP and other archives.

[encode]

Tested various exe-filters with PPMd. One of the best is simplest E8 transformer.

Some testing results:

reaktor.exe (14,446,592 bytes):

PIM 2.01 (alpha): 2,059,104 bytes
PIM 2.01 (alpha) + EXEFLT: 1,951,068 bytes

A nice gain. Maybe I should add this filter to PIM. Note that in this case you can control it's usage via settings dialog (Use EXE-filter option or something like that).

NOTE: IF I'LL ADD SUCH FEATURE, I'LL PROBABLY BREAK COMPATIBILITY WITH CURRENT ALPHA. BE PREPARED - DO NOT USE AN ALPHA VERSIONS - THEY INTEDED FOR TESTING PURPOSES ONLY!

[Vacon]

Hello everyone,

thank you for warning!
But breaking compatibility with an earlier ALPHA shouldn't be a problem. I personaly consider them as "proof-of-concept"-versions.
Breaking compatibility with earlier FINAL could be more serious...
Anyway: your plans sound interesting.

Best regards!

[encode]

Breaking compatibility with earlier FINAL could be more serious...

Note that the PIM v1 compatibility will be untouched. Its only about an alpha version - thats why I call it alpha...

[Vacon]

Hello everyone,

Note that the PIM v1 compatibility will be untouched.

Thats good news!

...thats why I call it alpha...

I guess "beta" and "alpha" nowadays get new definitions, as many programmers (Igor with 7-zip for instance) call their new releases "beta" allthough its allmost "final".
But with definition "alpha" all my red-lights go on ->
I need not to have _every_ experience myself...
Good luck with evoluting PIM 2!

[encode]

I guess I will keep compatibility. Instead of adding a new transformType field I added a new method called ExePPMd.

Currently I'm searching/testing for the best EXE-filter. Also I'm thinking how to call a new option in settings dialog. However, current test-filter works!

Check out a screenshot:

[Vacon]

Hello everyone,

since we are takling about the effect of evolution and compatibility:
what about an SFX-modul?
It could help archiving data and keep access even if next-generation programs loose compatibility (if OS supports the sfx-exe).

Bestr regards!

[encode]

Well, like I said, PIM2 is too young and too many features already in TODO list...

I'm working on compression, archive handling...

By the way, probably I'll add not just generic EXE-filter, but something advanced... (separate streams for calls/jumps, defragmentation, etc)

[Vacon]

Hello everyone,

was just a question!
Haste makes waste, so don't loose your aim!

Best regards!

[encode]

[encode]

In addition, I think I'll change the main compression from PPMd var. I to PPMd var J. (latest). This will break compatibiliry with latest alpha. Anyway, var. J has better comression!

[Black_Fox1]

Good idea... why does winzip use var. I, when there's (according to readme) a bug?

[encode]

why does winzip use var. I, when theres (according to readme) a bug?

I think when WinZip developers added PPMd, PPMd var J was still unreleased. - i.e. in that times var. I was the latest. For example, 7-Zip and RAR also uses old versions of PPMd.

[encode]

(Note that each version of PPMd is incompatible with others)

[encode]

Also, I'll probably remove ZIP PPMd decompression, to override chaos in PIM's source code. I think these days I should seriously focus on PIM's main compression. Using PPMd var.J + EXE filter will give PIM modern power since:
Naturally, all MM data already compressed - here, if PIM cannot compress it will just store a file.
PPMd shows supreme performance - it has really high text compression, and good binary compression. With an EXE transformer EXE compression even further improved.
PPMd works better with non-solid mode - since outdated statistics often hurts compression, especially if you compress lots of different files. This is one of the reason why PIM v2 uses non-solid mode.

So anyways, soon, I'll release a version with PPMd var. J. Later a new version with EXE-transformer. I just need make more experiments with EXEFLT to grab the best!

[Vacon]

Hello everyone,

...Ill probably remove ZIP PPMd decompression...

hmm, wouldnt that remove (at least partial) compatibility with WinZip 11 which was one of your aims? At least decompression could be a fine feature, or have I missed something? How wide is PPMd spread in zip-files / how many people use it actually?

Best regards!