LZ4X - An Optimized LZ4 Compressor

**encode** · 27th November 2016, 10:09

Updated to LZ4X v1.30!
The newest version features a much faster decoder. I replaced the lame byte-by-byte copy with a fast copy128, at last!
Enjoy new release!

**Stanley** · 30th November 2016, 15:11

Originally Posted by encode

License: Public Domain. Updated to LZ4X v1.30!
The newest version features a much faster decode.

When i reed you'r post that LZ4X is faster than original LZ4, and Your post about it's license (not GPL). My first thunk was: Did you already commited this peace of code for example to Oracle/ZFS source tree , to Linux -> Oracle/BTRFS source tree etc. BSD/Apple source tree etc. Because if you do - We all could be happy of your work..

Actually the great revolution (i think) in compression is Zstandard and new entropy algorithm invented in Poland (is i know, on some university?) standardized by Facebook (or maybe im mixing two different things). But still LZ4 is one of greatest "inventions" - useful compromise. So i hope (because i haven't reed all posts) You preserve compatibility with this standard. And I hope you commited (or will commit) to ZFS source tree?

**encode** · 26th February 2017, 00:24

Well, I think Optimal Parsing within original LZ4 package ruined the LZ4X idea.
Now I have an improved core of a fast bytewise LZ77 (ULZ/LZ4X).
My question is:
Look at the thread's poll

**encode** · 7th November 2018, 00:00

To revive my LZ4X, I (re)added the Brute mode (new -9) to the LZ4X once again, so now it compresses slightly higher than LZ4 -l -12 or smallz4 (<= 8 MB files). It will be released within a few weeks.

**jibz** · 7th November 2018, 00:08

Sounds great!

Do you know if the posts from the past two weeks will be back? there was some good discussion going on about optimal parsing in the BriefLZ thread I think.

**encode** · 10th November 2018, 12:35

Too much's going on in the LZ4 field so, please test this version instead:

**jibz** · 10th November 2018, 14:35

A quick test runs gives the exact same output size as blz4 --optimal on all files of the Canterbury and Silesia corpora, with much faster compression. Great job!

**introspec** · 13th November 2018, 00:39

I also did some tests (Calgary, Canterbury, selected Silesia files, as well as much my bunch of ZX Spectrum-related test files).
The resulting file sizes are all the same for lz4x ver.1.49b and for blz4 ver.0.1.0. I was not able to find a file on which the results will be different.
I am guessing lz4x is the new default .lz4 compressor for me now

**encode** · 14th November 2018, 11:48

Yet another beta to test. This version features speed improvements - some redundant computations were eliminated. The problem with the worst case with the match optimizer now solved - only the match finder may hang.

**encode** · 14th November 2018, 21:40

Okay, the LZ4X v1.50 has been released:
https://github.com/encode84/lz4x

**encode** · 26th March 2019, 20:46

Updated to v1.60! New version features overrun protection in the decoder + some small code changes!

https://github.com/encode84/lz4x

View Poll Results: What should I release next?

Thread: LZ4X - An Optimized LZ4 Compressor

Thread Tools

Search Thread

Display

Thanks (4):

ZFS, OPENZFS, BTRFS

Thanks:

Thanks (2):

Thanks (4):

Thanks:

Thanks:

Thanks (2):

Thanks (8):

Thanks (4):

Similar Threads

M1 - Optimized demo coder

LZF - Optimized LZF compressor

Optimized LZSS compressor

lzop optimized compile

7zip >> Sfx optimized - 23,7 kb

Tags for this Thread

Posting Permissions