lzpm 0.11 is here!

[encode]

Just follow to the new homepage!

http://lzpm.encode.su/

[Nania Francesco]

Very good job! Nice!

[LovePimple]

Thanks Ilia!

Another awesome release!

[Black_Fox1]

Thanks encode!

LZPM 0.08 __13 409 100____958 / 9 866
LZPM 0.09 __13 400 783___1 028 / 9 866
LZPM 0.10 __13 383 068____967 / 9 866
LZPM 0.11 __13 545 901____827 / 8 969

leaving exeflt out had some impact... 0.11 is generally slightly stronger than 0.10... one interesting thing is that TXT2 got compressed best at compression level 2 (difference in compression time to level 9 was much bigger than at other files), when all other files were squished best at level 9

[Fallon]

You've done lots and it's good!
Thank you!

[encode]

leaving exeflt out had some impact... 0.11 is generally slightly stronger than 0.10...

Mainly, the difference will be with large files >= 64 MB. The decompression speed is also slightly affected - its due to a larger ROLZ table (4 MB vs 16 MB). Just waiting for Large Text Benchmark results. Maybe LZPM should be slimdowned - 8K index instead of current 16K, 32 or 16 MB buffer, instead of 64 MB. Anyway, with such configuration the difference with ENWIK9 is huge, probably the decompression speed will be not affected or even faster with EINWIKs.

one interesting thing is that TXT2 got compressed best at compression level 2 (difference in compression time to level 9 was much bigger than at other files), when all other files were squished best at level 9

That means there is a flaw in parsing scheme. At present time I face only with the one file which compressed with other levels ("7") better than with "9". However, test carefully the ROLZ2 from MCOMP, and youll find that sometimes Normal outcompresses the Max mode.

Anyway, LZPM 0.11 is an experiment. You can compare it with previous versions and make your own verdict.

[Vacon]

Hello everyone,

simply great

Best regards!

[encode]

2 Black_Fox & others:
Here is the special build of LZPM 0.11 - LZPM 0.11lite:
lzpm011lite.zip

This one has 32 MB buffer and 8K index. Can you please test it and post the compression results and the decompression speed.

Thank you!

I just think that this variant is more adequate - it uses much less memory and much faster at compression. Just interesting to see its decompression speed, especially on Black_Fox's benchmark, since Radek tested at almost ALL versions of the LZPM, including their decompression speeds.

[LovePimple]

Thanks Ilia!

[encode]

LZPM 0.11 was tested on Large Text Benchmark. Thanks Matt!

Well, it decompresses as fast, introducing a new record in terms of compression ratio and decompression speed! (LZTURBO was nailed by LZPM)

[LovePimple]

Its awesome! I'm looking forward to the MC results.

[Nania Francesco]

The Best LZ Compressor!!

[Fallon]

Maybe LZPM should be slimdowned - 8K index instead of current 16K, 32 or 16 MB buffer, instead of 64 MB.

Is there not room for both versions? I like any archiver that will run on all PCs under the sun. Then it will be used sooner. But in that case other things will enter the equation as well, like how sophisticated is the command syntax for archiving? There is always something to improve.
When we keep in mind that in a few years time, we will all run 64-bit systems with many gigs of memory, then it also seems logical to go there already with experimental compressors.
Lzpm is ok either way, its just a choice.

Binary Tree is better with optimal parsing since the string search becomes faster. Just currently I have no idea how exactly use binary trees for string searching. Probably I should look at LZMA sources.

Do you plan to look into this at some point later?

[Black_Fox1]

This one has 32 MB buffer and 8K index. Can you please test it and post the compression results and the decompression speed.

Sure ___________________in / out
LZPM 0.10___13 383 068____967 / 9 866
LZPM 0.11___13 545 901____827 / 8 969
LZPM 0.11lite 13 536 090 __1 165 / 9 866

[squxe]

Thanx Ilia!
What about release addition version with EXE filter (something like lzpm 0.11exe)?

[encode]

What about release addition version with EXE filter (something like lzpm 0.11exe)?

The only reason I removed EXE filter is that this filter not works with buffer larger than 16 MB. Thus to add an exe filter I must write something new - will have an additional spare time, will thinking about that.

Sure ___________________in / out
LZPM 0.10___13 383 068____967 / 9 866
LZPM 0.11___13 545 901____827 / 8 969
LZPM 0.11lite 13 536 090 __1 165 / 9 866

Its what Im talking about - the decompression is faster with even higher compression.

Do you plan to look into this at some point later?

I do. However, with fast and, possibly, normal levels, hashing is the best anyway. So, it might be that LZPM will use two match finders as ROLZ2 do.
Recently, based on theory, Ive found the niche for ROLZ - its a fast compression (a fast large dictionary LZ, no more no less, as Malcolm described). Compared to the modern LZ77, like LZMA, ROLZ has some benefits with fast modes - it compresses faster and compression often better than plain LZ77 in fast modes. If you read some papers, including papers from Ross Williams, youll see that such one byte context with ROLZ helps in compression, especially with greedy or simple parsing schemes - i.e. even with greedy parsing we get "virtually" optimized parsing with no time loss, in addition, the baseline match finder works faster with ROLZ - since we search the longer strings initially.
Concluding, LZPM must be used with fast levels 1..3, while 9 - is just for benchmarking or for generating a tightest compression stream possible. So, Ill look at the results of LZPM 0.11, and, probably, Ill release a new 0.12 like "lite" version (LZPM 0.11lite) with some tunings.

[encode]

By the way, check out my new song at myspace:
http://www.myspace.com/djencode

[Matt Mahoney]

http://cs.fit.edu/~mmahoney/compression/text.html# 2291

[encode]

Thank you!

[encode]

2 Matt
Can you please test a special version of LZPM 0.11:
lzpm011lite.zip

Like I said, possibly this version is more adequate for practical use. Mainly I interested in the decompression time and in performance of followed levels:
1 - The fastest
3 - The most optimal in terms of compression time/ratio
9 - Max compression

[Matt Mahoney]

"lite" tests.
http://cs.fit.edu/~mmahoney/compression/text.html# 2291

[encode]

Thank you, Matt!

[giorgiotani]

Results are good, for lzpm011lite too. Both lzpm and lzpmlite seem interesting research threads!

[encode]

LZPM has been tested at MFC... These days I will think about the configuration of future versions. 16 MB buffer + 8K index + EXE filter should be OK...

[l1t]

I unzip
http://lzpm.encode.su/fp.log.lzpm
and
"C:Program Files7-Zip7z" a -t7z archive.7z fp.log -m0=PPMd -mx9
dir
2007-10-12 23:24 487,259 archive.7z

[l1t]

and
ccmx125 c 4 fp.log fp.ccm
2007-10-12 23:43 475,563 fp.ccm

[Black_Fox1]

-m0=PPMd

IMHO the value in the comparison table is obtained using LZMA, comparing LZPM with PPMd is unprobable (PPM is symmetric), same for CCM(x)... UHARC is in the comparison maybe because of its high compression ratio

[encode]

IMHO the value in the comparison table is obtained using LZMA, comparing LZPM with PPMd is unprobable (PPM is symmetric), same for CCM(x)... UHARC is in the comparison maybe because of its high compression ratio

Exactly! The catch of LZPM as with LZMA - fast decompression. PPMD is symmetric - i.e. the decompression takes about the same time as with the compression.

Maybe I should change UHARCs settings to ALZ:3?

[Black_Fox1]

It would be fair to use the same (asymmetric) method in all tests, but it should be also noted somewhere on the site