PIM 2.01 (beta) is here!

**encode** · 14th June 2007, 20:23

http://www.encode.su/

Enjoy!

**Vacon** · 14th June 2007, 21:16

Thank you Ilia!

**Nania Francesco** · 14th June 2007, 21:39

Good results in all file type! Best fo BMP!

**LovePimple** · 14th June 2007, 23:22

Awesome!

**squxe** · 14th June 2007, 23:30

Great!

BMP filter is one of the best!
What about adding in future special filter for WAV?

**encode** · 14th June 2007, 23:37

I tried a few WAV-filters, with PPMd they give just a small gain. Overall, it's not worth it. Note that PPMd (and all PPM-based algorithms) works not so good on analog/noisy data. Probably, some day, I'll add an advanced WAV filter or even specialized WAV compression.

**squxe** · 14th June 2007, 23:41

Cool! Good luck with your experiments!

**squxe** · 15th June 2007, 15:03

tested on http://maxcompress.narod.ru

**encode** · 15th June 2007, 15:59

Thank you!

**Matt Mahoney** · 17th June 2007, 01:40

Updated http://cs.fit.edu/~mmahoney/compression/text.html

**encode** · 17th June 2007, 10:31

Thank you Matt!

**Black_Fox1** · 17th June 2007, 16:37

Thanks encode! It performs generally better than v1.50 with exception of my Photoshop file, which gets compressed 1,7MB worse

Another question: Do all PNG saving programs use zlib compression (only those 4 or so possible outputted files)? I tried to compress my test PNG firstly as a PNG and secondly resaved to BMP -> PIM compressed png to 237kB and bmp to 136kB! So if there was a way to restore original PNG back from BMP, it could boost compression a lot sometimes

**encode** · 17th June 2007, 17:03

Originally Posted by Black_Fox

exception of my Photoshop file, which gets compressed 1,7MB worse

PIM 1.x make use of LZP. This file contains some sort of two copies of the same picture - LZ just copies first string, pure PPM cant do that.

Originally Posted by Black_Fox

So if there was a way to restore original PNG back from BMP, it could boost compression a lot sometimes

Many times I tested my algos against widespread lossless ones like GIF or PNG. My algos just more progressive compared to RLE, LZW or Deflate. Not sure about Deflate, but with RLE/LZW I think it is possible to restore bit-to-bit identical image without an extreme computation.

**encode** · 17th June 2007, 17:08

By the way, currently I'm working on 2.02 beta, along with some surprise features, I will add the most missed feature - opening file inside an archive - i.e. double-click on a file will automatically extracts and runs it. Just a question, where should PIM extract these files:
1. Current folder (which contains current archive)
2. Temp folder (standard Windows temp dir)

**Black_Fox1** · 17th June 2007, 17:52

Temp folder surely

And if you could add one more feature, please: When dropping file onto PIM's window with opened archive inside, don't "open archive", but "add file".

**encode** · 17th June 2007, 18:00

Originally Posted by Black_Fox

When dropping file onto PIMs window with opened archive inside, dont "open archive", but "add file".

When I add ability to add a file to an existing archive I will do so.

**encode** · 17th June 2007, 21:29

What I've done already with PIM v2.02:
+ I removed all junk/ballast features, including QUAD/QUAKE PAK archives support. Firstly, to increase the performance of the main format. Note that PIM is not bundled software like IZArc which has insane number of supported archive formats and hasn't own format. PIM is about own high-performance archive format. In other words it's about compression.
So, with new version I focused on main archive format. For example, I already rewrote the archive handling routines, including ZIP ones in favor to improve performance.
having said that I tried to add RAR and LHA support these days.
RAR - has too many features and add full support is not a trivial task. Note that all archive/compression routines was written by myself from scratch, including ZIP ones. I implemented the listing of a RAR archive - it works some how. After I refuse that.
LHA - has 3 level of headers. First was supported by old archivers, then an updated version of header became, and after again author changed specifications. So, now LHA is very strange and buggy format. I wrote a version for LHA with Level 0 headers all works fine, including my own implementation of CRC16 checksum used with LHA archives. In addition, I played with decompression for Store/Frozen-5 methods, again all works fine. Anyway, today Header-2 Level is standard (It's funny but WinAce creates LZH/LHA archives with Level 0 header). Finally, I drop this idea with LHA archives. Having said that compression of LHA is poor, plus CRC16 instead of CRC32, plus buggy specs...

Anyway, what I've done with PIM:
+ Removed options for enabling/disabling executable/true color compression. Each time PIM will automatically detect file type and perform algo selection.
+ Added TIFF file compression. Now PIM can read TIFF files (Both IBM and MAC types, and if picture in RGB 24-bit format use special compression). For example waterloo test-set consists of tiff files.
Results:
PIM 2.01: 4,573,834 bytes
PIM 2.02: 3,583,184 bytes
A nice gain, isn't it?

+ Added open file inside an archive feature, mentioned above.
+ Small GUI changes.
Continue working...

qqwertyy · 18th June 2007, 09:19

2616175616 CS counter-strike-1.6 custom_maps
1142055900 CS-1.6.pim pim-2.0.1b PPMd executable true_color
1103985468 CS-1.6.7z p7zip-4.47b 7za a -t7z -mx=9 -md=32m -ms=off

зы
Deflate64 is not supported in zip files

**squxe** · 18th June 2007, 11:49

Originally Posted by encode

+ Added TIFF file compression. Now PIM can read TIFF files (Both IBM and MAC types, and if picture in RGB 24-bit format use special compression)

coool!!!

**encode** · 18th June 2007, 12:16

Originally Posted by qqwertyy

Deflate64 is not supported in zip files

I will add the Deflate64 support to upcoming PIM 2.02!

**Alex** · 18th June 2007, 19:59

Nice job with PIM so far.

However, is it going to be possible to add files to existing archives in the near future? I can't manage to do that in PIM 2.01 beta.

**encode** · 18th June 2007, 20:27

Stuck in the middle with Deflate64!!!

Damn algo! To few info about this rare creature. 7-Zip uses custom implementation. I make use of modified zlib, but zlib wont support Deflate64, it has a patch named InfBack9, but callbacks... its buggy...

Originally Posted by Alex

However, is it going to be possible to add files to existing archives in the near future?

It is possible. Furthermore, I already tested such feature, it worked, but sometimes worked buggy. These days I will retest such thing!

**LovePimple** · 18th June 2007, 21:17

Originally Posted by encode

Damn algo! To few info about this rare creature

Maybe something here will be helpful!

http://www.planet-source-code.com/vb/scripts/Brows eCategoryOrSearchResults.asp?lngWId=1&blnAuthorSea rch=TRUE&lngAuthorId=3317271322&strAuthorName=Marc o%20v/d%20Berg&txtMaxNumberOfEntriesPerPage=25

http://www.pscode.com/vb/scripts/ShowCode.asp?txtC odeId=58473&lngWId=1

**encode** · 18th June 2007, 22:27

Thanks! I will additionally dig for info!

Info-Zip supports Deflate64.

By the way WinRAR uses Info-Zip.

**encode** · 18th June 2007, 22:34

X. Enhanced Deflating - Method 9
--------------------------------

The Enhanced Deflating algorithm is similar to Deflate but
uses a sliding dictionary of up to 64K. Deflate64(tm) is supported
by the Deflate extractor.

[This description is inofficial. It has been deduced by Info-ZIP from
close inspection of PKZIP 4.x Deflate64(tm) compressed output.]

The Deflate64 algorithm is almost identical to the normal Deflate algorithm.
Differences are:

- The sliding window size is 64k.

- The previously unused distance codes 30 and 31 are now used to describe
match distances from 32k-48k and 48k-64k.
Extra
Code Bits Distance
---- ---- -----------
.. .. ...
29 13 24577-32768
30 14 32769-49152
31 14 49153-65536

- The semantics of the "maximum match length" code #258 has been changed to
allow the specification of arbitrary large match lengths (up to 64k).
Extra
Code Bits Lengths
---- ---- ------
... .. ...
284 5 227-258
285 16 3-65538

Whereas the first two modifications fit into the framework of Deflate,
this last change breaks compatibility with Deflate method 8. Thus, a
Deflate64 decompressor cannot decode normal deflated data.

**encode** · 18th June 2007, 23:18

Found info how to use InflateBack/InflateBack9!

**LovePimple** · 19th June 2007, 01:44

Originally Posted by encode

Found info how to use InflateBack/InflateBack9!

Excellent!

**ariya** · 19th June 2007, 13:51

Originally Posted by encode

Found info how to use InflateBack/InflateBack9!

Very nice!

**encode** · 19th June 2007, 14:54

Victory!

Added Deflate64 support!

In addition, I optimized ZIP BZip2 decompression.

**LovePimple** · 19th June 2007, 15:21

Originally Posted by encode

Added Deflate64 support!

Thread: PIM 2.01 (beta) is here!

Thread Tools

Search Thread

Display

Similar Threads

PIM 2.40 BETA is here!

PIM v2.41 BETA is here!

PIM 1.25 beta is here!

PIM 1.20 beta is here!

PIM 1.05 beta is here!

Posting Permissions