CCM file compressor

[LovePimple]

A new file compressor by Christian Martelock...

Hi everyone!
Over the last few weeks Ive developed a tiny compressor called CCM. Id like to share it with you but I dont have any webspace.

CCM is a general purpose symmetric file compressor with good compression
ratio. It uses a VERY simple and hopefully fast CM-algorithm combined with
a "dirty" match-making hack.

Christian

Download CCM v1.0.2a
Download CCM v1.0.3a
Download CCM v1.0.4a
Download CCM v1.0.4a2
Download CCM v1.0.5a
Download CCM v1.0.5a2
Download CCM v1.0.5a3
Download CCM v1.0.5a4
Download CCM v1.0.6a
Download CCM v1.0.6a2

Now also available at Werners site

[LovePimple]

First public results for CCM extra:

A10.jpg > 834,775
AcroRd32.exe > 1,282,440
english.dic > 562,440
FlashMX.pdf > 3,678,634
FP.ccm > 485,546
MSO97.dll > 1,727,502
ohs.doc > 752,918
rafale.bmp > 819,586
vcfiu.hlp > 565,749
world95.txt > 530,436

Total = 11,240,026 bytes


PHT.psd > 1,118,977

Compression/decompression speeds were very quick!

[Nania Francesco]

enwik8 24.027.805

rafale.bmp 819.586
world95.txt 530.436
fp.log 485.546 t.a
english.dic 562.440
flashmx.pdf 3.678.634
mso97.dll 1.727.502
vcfiu.hlp 565.749
a10.jpg 834.775
acrord32.exe1.282.440
ohs.doc 752.918

Very good work! Good Speed!

[LovePimple]

enwik8 24.027.805

You just beat me to it with the ENWIK8 results!

How about ENWIK9?

[LovePimple]

Good work Christian!

I hope that you will work to improve compression ratio in future versions.

[encode]

Two questions:

1. Who is Christian Martelock? And how he witin a couple of weeks develop such amazing compressor? Must be he an extremenly experienced professional.

2. I'm very curious about implementation details of CCM! Does CCM uses LZ ("dirty" match-making hack)? And how Christian make (high-order) CM compression so fast?

[Christian]

Hi everyone!
Thank you very much for your feedback.

Good work Christian!

I hope that you will work to improve compression ratio in future versions.

Sadly in the next few month I do not have time to play around with data compression.
CCM as it is now is only a symbol coder. At this point improving compression is not very difficult because the symbol coder is so simple. If you add some bells and whistles to the CM or by just uping the memory usage (from 97 -> 200+ MiB) compression will improve a bit for sure. But my goal is to keep a reasonable speed and memory usage. So this is not an option.

CCM is not that good on very redundant files where long matches over longer distances are common (text compression, ...). If Ill add some sort of (RO)LZ/LZ(P) layer to the algorithm it should improve on these files a bit (at least I hope :confused. This will be the next step when Ill find some time.

See you around,
Christian

[Christian]

1. Who is Christian Martelock? And how he witin a couple of weeks develop such amazing compressor? Must be he an extremenly experienced professional.

Im not a professional. Im just toying with data compression as a hobby from time to time. Ive been looking into context mixing over the last few months. So a few weeks ago I decided to put some things together and created the compressor. Thats it.

2. Im very curious about implementation details of CCM! Does CCM uses LZ ("dirty" match-making hack)? And how Christian make (high-order) CM compression so fast?

It does not use any LZ but tries to model something similar (but weaker). If this works out it switches off some models and gains speed. The algorithm only uses 5 models. Only one can be described as being high order. This explains the speed and the mediocre performance on text data.

I hope this somehow makes sense to anyone. Btw., I really like your compressors and Im always curious about the progress you make. Note, I wonder how you make high order CM so fast in your latest TC.

Regards, Chris

[Black_Fox1]

fast profile - speed ~1500kB/s
extra profile- speed ~670kB/s

It's really nice program, results will be published later in the afternoon

[Christian]

fast profile - speed ~1500kB/s
extra profile- speed ~670kB/s

Its really nice program, results will be published later in the afternoon

Thank you! Since I developed the program using MC testset Im wondering how it will perform on your testset. Btw., over at MC Ive posted an updated version (1.0.3a) in the guestbook which makes improvements to the "fast profile". Id be nice if you use this version if youre going to test the fast profile, too.

Chris

[LovePimple]

Sadly in the next few month I do not have time to play around with data compression.

Hello Chris!

PLEASE consider returning to data compression sometime in the future and continue work on your amazing compression engine.

BTW: Could you use http://www.mooload.com/ for uploading any future updates. Mytempdir used to be a great file host, but its been hacked recently and as a result has become a little unreliable.

Kind Regards,
LovePimple

[LovePimple]

Btw., over at MC Ive posted an updated version (1.0.3a) in the guestbook which makes improvements to the "fast profile".

Thanks Chris! I have added a download link to my original post.

[Nania Francesco]

results CCM_EXTRA: ENWIK9 207.273.926 bytes

[LovePimple]

results CCM_EXTRA: ENWIK9 207.273.926 bytes

Thanks Nania!

[Christian]

Hello Chris!

PLEASE consider returning to data compression sometime in the future and continue work on your amazing compression engine.

Its just that I dont have time the next few months. Since compression is a hobby of me Ill still be working on some things from time to time. You know, an idea can sound great on paper, but is useless if you try it out. This takes time. If Im up to something new and have implemented it Ill share it with you guys.

@all: Thank you for the warm welcome! Compression scene is a nice place to be.

Chris

[Christian]

Hi everyone!

I hope I don't bug you with so many new versions. But I've made one more little improvement. This improves a little bit on text files and stuff. Speed dropped a few KiB/s and memory usage is up 3-5%. I hope you like it.

Again on mytempdir because mooload doesn't work for me:

http://www.mytempdir.com/1213449

For substantial changes you'll have to wait some more months.

Chris

[Nania Francesco]

fp.log 456.679
rafale.bmp 818.621
world95.txt 517.251
english.dic 559.752
flashmx.pdf 3.679.595
mso97.dll 1.727.413
vcfiu.hlp 564.683
a10-jpg 834.777
acrord32.exe 1.283.006
ohs.doc 752.879
TOTAL 11.194.656 !!!
ENWIK8 -> 23.407.606 Bytes

[Nania Francesco]

Cristan I wanted to signal a bug to you! If it compress one file nonexistent it goes in crush!

[Nania Francesco]

Here results on ENWIK9 : 201.132.358 Bytes
Good work Christian!!!

[Christian]

Last tiny update for today. This is version 1.0.4a2.

These are the two changes:
-fixed crashs when a file doesnt exist, ...
-changed modeling a bit again => little bit better results on MC and ENWIK

http://www.mytempdir.com/1213882

Cristan I wanted to signal a bug to you! If it compress one file nonexistent it goes in crush!

Thanks for that. I knew someone would find out that Im too lazy to implement things like this.

Here results on ENWIK9 : 201.132.358 Bytes
Good work Christian!!!

Thanks!

Regards, Chris

[Nania Francesco]

world95.txt 516.343
fp.log 449.915
rafale.bmp 818.621
english.dic 559.752
flashmx.pdf 3.679.286
mso97.dll 1.727.384
vcfiu.hlp 563.268
a10.jpf 834.777 t.a
acrord32.exe 1.283.630
ohs.doc 754.403
ENWIK8 : 23.329.823 Bytes
They are curious to see it in action with 1 GB of memory!
We hope that a day you it can put Open Source!

[encode]

CCM at Source Forge!

[LovePimple]

Hi everyone!

I hope I dont bug you with so many new versions.

This will NEVER happen!!!

Im sure that everone here will agree with me in inviting you to post as many updates (or other work) as you like.

[encode]

Im sure that everone here will agree with me in inviting you to post as many updates (or other work) as you like.

[Christian]

They are curious to see it in action with 1 GB of memory!

It just doesnt feel right to use such a big amount of memory. On the other hand it will not help much.

We hope that a day you it can put Open Source!

As long as Im developing on it itll definitly remain closed source. So this will not happen in the near future since the algorithm is so very alpha.

I got a new version 1.0.5a here. Im curious what you say about it.
http://www.mytempdir.com/1214530

Chris

[encode]

File uploaded to encode.su!

First results:

A10.jpg: 834,749 bytes
acrord32.exe: 1,269,049 bytes
english.dic: 559,436 bytes
FlashMX.pdf: 3,676,807 bytes
fp.log: 450,575 bytes
mso97.dll: 1,689,155 bytes
ohs.doc: 754,369 bytes
rafale.bmp: 818,029 bytes
vcfiu.hlp: 563,081 bytes
world95.txt: 514,631 bytes

Total: 11,129,881 bytes

Impressive!

P.S.
CAN'T WAIT FOR MAXIMUMCOMPRESSION.COM UPDATE!

[Christian]

This one now has EXE-detection. Improves a VERY little bit on most files. Perhaps not worth the lines in the code but now it stays.

http://www.mytempdir.com/1214659

Chris

[encode]

Thank you!

[Nania Francesco]

CCM_EXTRA 1.05A2
fp.log 450.575
rafale.bmp 815.344
world95.txt 514.631
english.dic 559.436
flashmx.pdf 3.676.751
mso97.dll 1.690.582
vcfiu.hlp 562.651
a10.jpg 834.697
acrord32.exe 1.267.577
ohs.doc 752.823
TOTAL: 11.125.067 (FANTASTIC!!!!!!!!!!)
ENWIK8 23.329.212 (NEED MORE MEMORY?)
Calgary c. (concatenated ISO)= 764.035
Canterbury c.(conc. ISO) = 447.215

[Nania Francesco]

ENWIK9: 200.214.411 Bytes