LZHAM

**m^2** · 17th August 2010, 10:02

I just stumbled upon a codec that does something I've been thinking to do:

LZHAM (LZ, Huffman, Arithmetic, Markov) is a data compression library that borrows many ideas from LZMA but purposely makes several key tradeoffs that favor decompression speed over compression ratio. LZHAM's compression ratio is approximately 1.5% less than LZMA, but decompresses 3-4x faster on a Core i7.

Sadly, I can't test it, there are 2 executables, one tries to import some kernel32.dll function which doesn't exist in XP and the other just shows something like "Hello I'm a memory->memory compressor. Tested, everything's OK." Yeah, right.

https://code.google.com/p/lzham/

**Vacon** · 17th August 2010, 11:10

Hello everyone,

Originally Posted by m^2

I just stumbled upon a codec that does something I've been thinking to do:
Sadly, I can't test it, there are 2 executables, one tries to import some kernel32.dll function which doesn't exist in XP and the other just shows something like "Hello I'm a memory->memory compressor. Tested, everything's OK." Yeah, right.

Is there some special reason that you didn't provide a link?

Best regards!

**Bulat Ziganshin** · 17th August 2010, 11:10

you forget the link: http://code.google.com/p/lzham/

**m^2** · 17th August 2010, 11:19

As Bulat guessed, I forgot. ^^
Updated the 1st post.

**m^2** · 17th August 2010, 11:37

Compiled it to work on XP.
Usage:

Code:

lzhamtest_x64 c input_file compressed_file
lzhamtest_x64 d compressed_file decompressed_file

Bookstar:

Code:

codec        mode      size     decompression time (CPU)
LZHAM 0x1001 uber      10893224 0.906 s.
7za 4.63     mx=9      10744976 1.656 s.
Arc 0.666    tor:10    11972555 1.390 s.
Arc 0.666    tor:10:c3 11999793 1.093 s.

Update: added Tornado to the mix. mode 10 was stronger than 11.

**Vacon** · 17th August 2010, 12:38

Hello everyone,

Originally Posted by m^2

Compiled it to work on XP.
Usage:

Code:

lzhamtest_x64 c input_file compressed_file
lzhamtest_x64 d compressed_file decompressed_file

Bookstar:

Code:

codec        mode size     decompression time (CPU)
LZHAM 0x1001 uber 10893224 0.906 s.
7za 4.63     mx=9 10744976 1.656 s.

Thank you!

Best regards!

**Vacon** · 17th August 2010, 15:53

Hello everyone,

I made some quick tests. Since LZHAM is a single-file compressor I tar-ed my DoubleCommander-folder first and compared LZHAM to 7z.
LZHAM:

Code:

Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

I:\Tmp\lzham>lzhamtest_x86_default c K:\win7PE\Programs\doublecmd.tar test.lzham

Dynamically loading DLL "I:\Tmp\lzham\lzham_x86.dll"
Loaded LZHAM DLL version 0x1001
Testing: Streaming compression
[...]
Success
Input file size: 44443136, Compressed file size: 11830544, Ratio: 73.38%
Compression time: 274.63
Consumption rate:  161832.1 bytes/sec, Emission rate:   43078.9 bytes/sec
Input file adler32: 0xD1B99793

7-Zip:

Code:

I:\Tmp\lzham>timer 7za a test.lzma K:\win7PE\Programs\doublecmd.tar
Timer 9.01 : Igor Pavlov : Public domain : 2009-05-31

7-Zip (A) 9.15 beta  Copyright (c) 1999-2010 Igor Pavlov  2010-06-20
Scanning

Creating archive test.lzma

Compressing  doublecmd.tar

Everything is Ok

Kernel Time  =     0.578 =    0%
User Time    =    65.593 =   97%
Process Time =    66.171 =   98%
Global Time  =    66.981 =  100%

Where file-size is -> 11.524.366 Bytes
Unpacking
LZHAM:

Code:

I:\Tmp\lzham>lzhamtest_x86_default d test.lzham test
Dynamically loading DLL "I:\Tmp\lzham\lzham_x86.dll"
Loaded LZHAM DLL version 0x1001
Testing: Streaming decompression
Success
Compressed file size: 11830552, Decompressed file size: 44443136
Decompression time: 3.27
Consumption rate: 3623446.2 bytes/sec, Decompression rate: 13611986.5 bytes/sec
Decompressed adler32: 0xD1B99793

7-Zip

Code:

I:\Tmp\lzham>I:\Tmp\lzham>timer 7za x test.lzma
Timer 9.01 : Igor Pavlov : Public domain : 2009-05-31

7-Zip (A) 9.15 beta  Copyright (c) 1999-2010 Igor Pavlov  2010-06-20

Processing archive: test.lzma

Extracting  doublecmd.tar

Everything is Ok

Size:       44443136
Compressed: 11524366

Kernel Time  =     0.375 =    8%
User Time    =     2.078 =   46%
Process Time =     2.453 =   54%
Global Time  =     4.500 =  100%

I:\Tmp\lzham>

Best regards!

**nanoflooder** · 17th August 2010, 17:30

Some more tests:

Intel T7200 detuned to 1.00 GHz (for cooling purposes)
test.tar, 283 MB (Mathcad 14 distribution)
No code has to be inserted here.

**m^2** · 17th August 2010, 19:32

Another file, installer (the one I used for testing century ago):
No code has to be inserted here.

Also, I updated the previous post with Tor results.

**m^2** · 18th August 2010, 14:10

Bulat, what do you think about adding LZHAM to FreeArc at some point?

**Bulat Ziganshin** · 18th August 2010, 15:21

i think that such compressor is useful but wait for better solution

**m^2** · 18th August 2010, 15:43

So you don't find it as amazing as I do?

**Fu Siyuan** · 18th August 2010, 17:38

Originally Posted by m^2

So you don't find it as amazing as I do?

I think the shining point of LZHAM is it creatively achives high ratio by Huffman-like coder.While LZX is obsoleted and LZMA is LZ+Ari which decompresses slower.
However, I think it isn't matured enough, should have potential to be improved. There should be no reason that LZHAM is slower in compression than LZMA since LZHAM use similar ideas.

**Fu Siyuan** · 20th August 2010, 05:25

http://www.ezcodesample.com/

I found this site very practical and interesting, it's the reference mentioned in LZHAM site.

**Matt Mahoney** · 27th August 2010, 05:59

Tested. http://mattmahoney.net/dc/text.html#2246

I found the idea of Polar codes interesting. There is a simple algorithm to calculate them and the compression is only slightly worse than Huffman codes.

**m^2** · 27th August 2010, 10:12

Matt, can you test different LZHAM modes too?
You can find the execs attached here.

**zody** · 27th August 2010, 16:08

New Alpha2 is here.. click me

**m^2** · 27th August 2010, 17:15

XP compatible compiles, also with modes other than default.
ADDED:
Added modes that use more memory.

**m^2** · 27th August 2010, 18:07

TCUP:
No code has to be inserted here.
Exe:
No code has to be inserted here.Quick tests. Seems that compression is unchanged

**Matt Mahoney** · 30th August 2010, 20:43

lzham alpha 3.6 test program supports all of the library options. The max options improve compression without much speed penalty. http://mattmahoney.net/dc/text.html#2140

**m^2** · 30th August 2010, 22:13

Originally Posted by Matt Mahoney

lzham alpha 3.6 test program supports all of the library options. The max options improve compression without much speed penalty. http://mattmahoney.net/dc/text.html#2140

lzham deserves an underscore now.

Nothing stronger decompresses faster.
Idea for LTCB:
How about treating memory needed for compression and decompression independently?

**Vacon** · 31st August 2010, 01:28

Hello everyone,

Originally Posted by m^2

lzham deserves an underscore now.

[...]

well, it's on it's way.
I found some things to mention:
- the alpha 3 runs on XP out-of-the-box (so m^2 is un-employed now

)
- if I run "lzhamtest_x86 a test-folder" a temporary file is made that vanishes after compression, but when I give a name to create the archive -> "lzhamtest_x86 a test-folder test.lzham" it complains "Error: too many filenames!". Any hints? Am I too tired to see the right command?

Best regards!

**Matt Mahoney** · 31st August 2010, 03:01

Yes, you're right. lzham is on the Pareto frontier for decompression time. Probably should list memory separately for decompression but don't feel like retesting everything.

**spark** · 31st August 2010, 05:57

Originally Posted by Matt Mahoney

lzham alpha 3.6 test program [...] http://mattmahoney.net/dc/text.html#2140

Matt, a minor point.

On LTCB you refer to "lzham alpha 2.7". Looking at the official LZHAM pages, I don't see any such version.

Maybe you confused the ".7" from the beginning of the 7zip file extension of the distribution archive ("lzham_alpha2.7z") as being part of the version number?

Initially, I thought that your reference to "alpha 3.6" was also an error/typo but you could be referring to a build of alpha3 revision6 sources, yeah?

**joerg** · 31st August 2010, 09:35

lzham - a rising sun with the capabilities of the 7z - archivfile - format ?

@vacon: "Am I too tired to see the right command?"

for me it seems

the switch "a" = Recursively compress all files under "inpath" is only for test purposes

there is also not a switch "x" = extract inclusively folder

@m^2: "so m^2 is un-employed now"

can you fix this problem ? (not deleting the resulting file)

best regards

**m^2** · 31st August 2010, 10:19

Originally Posted by joerg

@m^2: "so m^2 is un-employed now"

can you fix this problem ? (not deleting the resulting file)

best regards

Sorry, no. The XP fix was removal 4 bytes from one file in pretty obvious places. Fixing a regular bug requires some understanding and this is less critical and I'm not willing to do it now.

**m^2** · 31st August 2010, 15:02

The LZHAM's author run LTCB with 512 MB dictionary.
Click.

**Matt Mahoney** · 1st September 2010, 22:06

You're right about the version numbers. I included the author's test results. http://mattmahoney.net/dc/text.html#2065

**Vacon** · 3rd September 2010, 20:18

Hello everyone,

I have now tried several times to file an issue for lzham (see above -> "a"-switch), but everytime I submit it I get "400"-error:

Bad Request
Your client has issued a malformed or illegal request.

Has anyone else tried to get in contact with the author?

Best regards!

**m^2** · 7th September 2010, 16:34

Alpha 4, supposedly stronger and faster, released.

Thread: LZHAM

Thread Tools

Search Thread

Display

LZHAM

Thanks:

Posting Permissions