Page 1 of 2 12 LastLast
Results 1 to 30 of 55

Thread: CCM file compressor

  1. #1
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    A new file compressor by Christian Martelock...

    Quote Originally Posted by Christian Martelock
    Hi everyone!
    Over the last few weeks Ive developed a tiny compressor called CCM. Id like to share it with you but I dont have any webspace.

    CCM is a general purpose symmetric file compressor with good compression
    ratio. It uses a VERY simple and hopefully fast CM-algorithm combined with
    a "dirty" match-making hack.

    Christian
    Download CCM v1.0.2a
    Download CCM v1.0.3a
    Download CCM v1.0.4a
    Download CCM v1.0.4a2
    Download CCM v1.0.5a
    Download CCM v1.0.5a2
    Download CCM v1.0.5a3
    Download CCM v1.0.5a4
    Download CCM v1.0.6a
    Download CCM v1.0.6a2

    Now also available at Werners site

  2. #2
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    First public results for CCM extra:

    A10.jpg > 834,775
    AcroRd32.exe > 1,282,440
    english.dic > 562,440
    FlashMX.pdf > 3,678,634
    FP.ccm > 485,546
    MSO97.dll > 1,727,502
    ohs.doc > 752,918
    rafale.bmp > 819,586
    vcfiu.hlp > 565,749
    world95.txt > 530,436

    Total = 11,240,026 bytes


    PHT.psd > 1,118,977

    Compression/decompression speeds were very quick!

  3. #3
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts
    enwik8 24.027.805

    rafale.bmp 819.586
    world95.txt 530.436
    fp.log 485.546 t.a
    english.dic 562.440
    flashmx.pdf 3.678.634
    mso97.dll 1.727.502
    vcfiu.hlp 565.749
    a10.jpg 834.775
    acrord32.exe1.282.440
    ohs.doc 752.918

    Very good work! Good Speed!

  4. #4
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by Nania Francesco Antonio
    enwik8 24.027.805
    You just beat me to it with the ENWIK8 results!

    How about ENWIK9?

  5. #5
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Good work Christian!

    I hope that you will work to improve compression ratio in future versions.

  6. #6
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    Two questions:

    1. Who is Christian Martelock? And how he witin a couple of weeks develop such amazing compressor? Must be he an extremenly experienced professional.

    2. I'm very curious about implementation details of CCM! Does CCM uses LZ ("dirty" match-making hack)? And how Christian make (high-order) CM compression so fast?


  7. #7
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 153 Times in 18 Posts
    Hi everyone!
    Thank you very much for your feedback.

    Quote Originally Posted by LovePimple
    Good work Christian!

    I hope that you will work to improve compression ratio in future versions.
    Sadly in the next few month I do not have time to play around with data compression.
    CCM as it is now is only a symbol coder. At this point improving compression is not very difficult because the symbol coder is so simple. If you add some bells and whistles to the CM or by just uping the memory usage (from 97 -> 200+ MiB) compression will improve a bit for sure. But my goal is to keep a reasonable speed and memory usage. So this is not an option.

    CCM is not that good on very redundant files where long matches over longer distances are common (text compression, ...). If Ill add some sort of (RO)LZ/LZ(P) layer to the algorithm it should improve on these files a bit (at least I hope :confused. This will be the next step when Ill find some time.

    See you around,
    Christian

  8. #8
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 153 Times in 18 Posts
    Quote Originally Posted by encode
    1. Who is Christian Martelock? And how he witin a couple of weeks develop such amazing compressor? Must be he an extremenly experienced professional.
    Im not a professional. Im just toying with data compression as a hobby from time to time. Ive been looking into context mixing over the last few months. So a few weeks ago I decided to put some things together and created the compressor. Thats it.

    Quote Originally Posted by encode
    2. Im very curious about implementation details of CCM! Does CCM uses LZ ("dirty" match-making hack)? And how Christian make (high-order) CM compression so fast?
    It does not use any LZ but tries to model something similar (but weaker). If this works out it switches off some models and gains speed. The algorithm only uses 5 models. Only one can be described as being high order. This explains the speed and the mediocre performance on text data.

    I hope this somehow makes sense to anyone. Btw., I really like your compressors and Im always curious about the progress you make. Note, I wonder how you make high order CM so fast in your latest TC.

    Regards, Chris

  9. #9
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    fast profile - speed ~1500kB/s
    extra profile- speed ~670kB/s

    It's really nice program, results will be published later in the afternoon

  10. #10
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 153 Times in 18 Posts
    Quote Originally Posted by Black_Fox
    fast profile - speed ~1500kB/s
    extra profile- speed ~670kB/s

    Its really nice program, results will be published later in the afternoon
    Thank you! Since I developed the program using MC testset Im wondering how it will perform on your testset. Btw., over at MC Ive posted an updated version (1.0.3a) in the guestbook which makes improvements to the "fast profile". Id be nice if you use this version if youre going to test the fast profile, too.

    Chris

  11. #11
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by Christian
    Sadly in the next few month I do not have time to play around with data compression.


    Hello Chris!

    PLEASE consider returning to data compression sometime in the future and continue work on your amazing compression engine.

    BTW: Could you use http://www.mooload.com/ for uploading any future updates. Mytempdir used to be a great file host, but its been hacked recently and as a result has become a little unreliable.

    Kind Regards,
    LovePimple

  12. #12
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by Christian
    Btw., over at MC Ive posted an updated version (1.0.3a) in the guestbook which makes improvements to the "fast profile".
    Thanks Chris! I have added a download link to my original post.

  13. #13
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts
    results CCM_EXTRA: ENWIK9 207.273.926 bytes

  14. #14
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by Nania Francesco Antonio
    results CCM_EXTRA: ENWIK9 207.273.926 bytes
    Thanks Nania!

  15. #15
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 153 Times in 18 Posts
    Quote Originally Posted by LovePimple

    Hello Chris!

    PLEASE consider returning to data compression sometime in the future and continue work on your amazing compression engine.
    Its just that I dont have time the next few months. Since compression is a hobby of me Ill still be working on some things from time to time. You know, an idea can sound great on paper, but is useless if you try it out. This takes time. If Im up to something new and have implemented it Ill share it with you guys.

    @all: Thank you for the warm welcome! Compression scene is a nice place to be.

    Chris

  16. #16
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 153 Times in 18 Posts
    Hi everyone!

    I hope I don't bug you with so many new versions. But I've made one more little improvement. This improves a little bit on text files and stuff. Speed dropped a few KiB/s and memory usage is up 3-5%. I hope you like it.

    Again on mytempdir because mooload doesn't work for me:

    http://www.mytempdir.com/1213449

    For substantial changes you'll have to wait some more months.

    Chris

  17. #17
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts
    fp.log 456.679
    rafale.bmp 818.621
    world95.txt 517.251
    english.dic 559.752
    flashmx.pdf 3.679.595
    mso97.dll 1.727.413
    vcfiu.hlp 564.683
    a10-jpg 834.777
    acrord32.exe 1.283.006
    ohs.doc 752.879
    TOTAL 11.194.656 !!!
    ENWIK8 -> 23.407.606 Bytes

  18. #18
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts
    Cristan I wanted to signal a bug to you! If it compress one file nonexistent it goes in crush!

  19. #19
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts
    Here results on ENWIK9 : 201.132.358 Bytes
    Good work Christian!!!

  20. #20
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 153 Times in 18 Posts
    Last tiny update for today. This is version 1.0.4a2.

    These are the two changes:
    -fixed crashs when a file doesnt exist, ...
    -changed modeling a bit again => little bit better results on MC and ENWIK

    http://www.mytempdir.com/1213882

    Quote Originally Posted by Nania Francesco Antonio
    Cristan I wanted to signal a bug to you! If it compress one file nonexistent it goes in crush!
    Thanks for that. I knew someone would find out that Im too lazy to implement things like this.

    Quote Originally Posted by Nania Francesco Antonio
    Here results on ENWIK9 : 201.132.358 Bytes
    Good work Christian!!!
    Thanks!

    Regards, Chris

  21. #21
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts
    world95.txt 516.343
    fp.log 449.915
    rafale.bmp 818.621
    english.dic 559.752
    flashmx.pdf 3.679.286
    mso97.dll 1.727.384
    vcfiu.hlp 563.268
    a10.jpf 834.777 t.a
    acrord32.exe 1.283.630
    ohs.doc 754.403
    ENWIK8 : 23.329.823 Bytes
    They are curious to see it in action with 1 GB of memory!
    We hope that a day you it can put Open Source!

  22. #22
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    CCM at Source Forge!

  23. #23
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by Christian
    Hi everyone!

    I hope I dont bug you with so many new versions.
    This will NEVER happen!!!

    Im sure that everone here will agree with me in inviting you to post as many updates (or other work) as you like.

  24. #24
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    Quote Originally Posted by LovePimple
    Im sure that everone here will agree with me in inviting you to post as many updates (or other work) as you like.

  25. #25
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 153 Times in 18 Posts
    Quote Originally Posted by Nania Francesco Antonio
    They are curious to see it in action with 1 GB of memory!
    It just doesnt feel right to use such a big amount of memory. On the other hand it will not help much.

    Quote Originally Posted by Nania Francesco Antonio
    We hope that a day you it can put Open Source!
    As long as Im developing on it itll definitly remain closed source. So this will not happen in the near future since the algorithm is so very alpha.

    I got a new version 1.0.5a here. Im curious what you say about it.
    http://www.mytempdir.com/1214530

    Chris

  26. #26
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    File uploaded to encode.su!

    First results:

    A10.jpg: 834,749 bytes
    acrord32.exe: 1,269,049 bytes
    english.dic: 559,436 bytes
    FlashMX.pdf: 3,676,807 bytes
    fp.log: 450,575 bytes
    mso97.dll: 1,689,155 bytes
    ohs.doc: 754,369 bytes
    rafale.bmp: 818,029 bytes
    vcfiu.hlp: 563,081 bytes
    world95.txt: 514,631 bytes

    Total: 11,129,881 bytes

    Impressive!

    P.S.
    CAN'T WAIT FOR MAXIMUMCOMPRESSION.COM UPDATE!

  27. #27
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 153 Times in 18 Posts
    This one now has EXE-detection. Improves a VERY little bit on most files. Perhaps not worth the lines in the code but now it stays.

    http://www.mytempdir.com/1214659

    Chris

  28. #28
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    Thank you!

  29. #29
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts
    CCM_EXTRA 1.05A2
    fp.log 450.575
    rafale.bmp 815.344
    world95.txt 514.631
    english.dic 559.436
    flashmx.pdf 3.676.751
    mso97.dll 1.690.582
    vcfiu.hlp 562.651
    a10.jpg 834.697
    acrord32.exe 1.267.577
    ohs.doc 752.823
    TOTAL: 11.125.067 (FANTASTIC!!!!!!!!!!)
    ENWIK8 23.329.212 (NEED MORE MEMORY?)
    Calgary c. (concatenated ISO)= 764.035
    Canterbury c.(conc. ISO) = 447.215

  30. #30
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts
    ENWIK9: 200.214.411 Bytes

Page 1 of 2 12 LastLast

Similar Threads

  1. FLASHZIP file Compressor!
    By Nania Francesco in forum Data Compression
    Replies: 170
    Last Post: 7th December 2013, 00:35
  2. lrzip a file compressor for linux
    By joerg in forum Data Compression
    Replies: 2
    Last Post: 9th December 2009, 15:20
  3. Blizzard - Fast BWT file compressor!!!
    By LovePimple in forum Data Compression
    Replies: 40
    Last Post: 6th July 2008, 15:48
  4. Fpaq0pv3 file Compressor
    By Nania Francesco in forum Forum Archive
    Replies: 13
    Last Post: 7th April 2008, 18:20
  5. FPAQ file compressor
    By LovePimple in forum Forum Archive
    Replies: 5
    Last Post: 2nd November 2006, 00:48

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •