CSC 3.1 is here!

[Fu Siyuan]

Hello all!

I wrote much in the new thread however I misclick the 'close button'.

Brief introduction this time:

New version is based on CSC 3. And I made some changes,mainly HashChain-4byte changed
to HashChain-6byte.So it can find longer matches in less search cycles. And the coder
now is using bit operation to handle bits with P=0.5 (like extra bits of match distance).
So decompression is faster. I added a simple detector. It calcuate entropy of different
channels of each small block. And use delta in some situation.However,the delta coding
scheme is so bad so it works poor on audio/bmp, but better than nothing. E89 transforming
also be used when necessary. The decompression process now checks the data, so it has less
chance to crash without warning message.
These changes leads to big progress: Now it ranked 4 on CompressionRatings .

Thanks to all on the forum who helped me and Sami Runsas!

[Matt Mahoney]

Nice improvement in speed and compression.

http://mattmahoney.net/dc/uiq/
http://mattmahoney.net/dc/text.html#2502

[PiPPoNe92]

good! the compression ratio and speed are improved!! thanks Fu Siyuan. Why you don't increase RAM memory required for compressing? So you can improve enaugh compression.

[Fu Siyuan]

good! the compression ratio and speed are improved!! thanks Fu Siyuan. Why you don't increase RAM memory required for compressing? So you can improve enaugh compression.

There are two points:

1. My destination is a compressor with good ratio / fast speed / and reasonable memory usage. In another word, I want it to be practical. So I think even if 772M (-d7) is too much, for there are still many computers with only 1G RAM or less.

2. I used LZ77 with Hash Chain Match finder. This means I can't use as much memory as I can. It won't improve much. Even the binary tree match finder, 10*Dictionary Size is enough. Statistical model can cost much memory, but I want to make it simple to achieve high speed decompression.

3. Binary tree match finder is a final idea to achieve best ratio of LZ77. I had once tried it , though not well programmed, it's two slow for me. Today, I carefully checked the datas on CompressionRating.com, I found that match finder isn't so important as I think if it does't be with an good parsing.

So what's the ultra asymmetric compressor? In my opinion, 7z is perfect, and if it was combined with filters, it would be the king asymmetric compressor. Fortunately, Bulat has done all these in his FreeARC.

[PiPPoNe92]

Ok Fu Siyuan!!!

P.S. "7z is perfect, and if it was combined with filters" What would you say? Which filters?

[Fu Siyuan]

Ok Fu Siyuan!!!

P.S. "7z is perfect, and if it was combined with filters" What would you say? Which filters?

As you can see on compressionratings.com, 7z performs bad on Audio/BMPs, or some data tables. It can not detect file type either. For example, if you rename a exe file to another extension, 7z would not use BCJ to preprocess it. Most mainstream archivers have the detectors and filters. If check the performance of nanozip -cD, you may find its LZ part is very weak, IMO, just because it has perfect detector and filters.

If you transform enwik8 into another file: make each char=255-char, then the DICT of nanozip won't work. And the real ratio on ENWIK8 of -cD is more than 33%.

[PiPPoNe92]

woow this is a great info!! thanks m8!

[Nania Francesco]

Very good release this CSC3.1 !
Nice results in MOC (Next Release) ! Super LZ77!