Page 2 of 3 FirstFirst 123 LastLast
Results 31 to 60 of 68

Thread: CCM 1.3x

  1. #31
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,583
    Thanks
    234
    Thanked 160 Times in 90 Posts
    Thanks Magic Cristian ! Hi!

  2. #32
    Member
    Join Date
    Feb 2009
    Location
    USA
    Posts
    58
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Test System: AMD Athlon 64 3000+ (Venice, single core) with 2GIG Corsair DDR CL2 RAM

    ccm130a-3.4.6 c 3 enwik8 (22351098 Bytes) - 1 m 28 s -
    ccm130a-3.4.6 d enwik8 - 1 m 25 s -
    ccm130a-4.1.2 c 3 enwik8 (22351098 Bytes) - 1 m 29 s - compression 1.1% slower
    ccm130a-4.1.2 d enwik8 - 1 m 29 s - decompression 4.7% slower
    -------------
    ccm130a-3.4.6 c 3 enwik8 (21646059 Bytes) - 1 m 53 s -
    ccm130a-3.4.6 d enwik8 - 1 m 53 s -
    ccm130a-4.1.2 c 3 enwik8 (21646059 Bytes) - 1 m 56 s - compression 2.6% slower
    ccm130a-4.1.2 d enwik8 - 1 m 58 s - decompression 4.4% slower
    --------------
    ccm130a-3.4.6 c 7 enwik8 (21980533 Bytes) - 1 m 29 s -
    ccm130a-3.4.6 d enwik8 - 1 m 30 s -
    ccm130a-4.1.2 c 7 enwik8 (21980533 Bytes) - 1 m 32 s - compression 3.3% slower
    ccm130a-4.1.2 d enwik8 - 1 m 34 s - decompression 4.4% slower
    --------------
    ccm130a-3.4.6 c 7 enwik8 (20857925 Bytes) - 1 m 59 s -
    ccm130a-3.4.6 d enwik8 - 2 m 0 s -
    ccm130a-4.1.2 c 7 enwik8 (20857925 Bytes) - 2 m 1 s - compression 1.6% slower
    ccm130a-4.1.2 d enwik8 - 2 m 2 s - decompression 1.6% slower

    Yeah, looks like 4.1.2 is not the best compiler to use in this case.

    Quote Originally Posted by Christian
    I rather prefer looking into changes/improvements to the algorithm than tweaking code.
    I agree completely. Youve got a lot more performance that way than by code tweaking.

    Thanks for your hard work!

  3. #33
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,583
    Thanks
    234
    Thanked 160 Times in 90 Posts
    Please you write the link for download the installer of gcc 3.4.6 !

  4. #34
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 163 Times in 18 Posts
    Quote Originally Posted by toffer
    I wonder how you do weighting without a dot product?! (or some vector math)
    Actually, I do not have one big mixer, but several small ones. They create some kind of mixing-tree. I simply cant update them at the same time because they all use different approaches (static, dynamic or table driven).

    Quote Originally Posted by Nania Francesco Antonio
    Please you write the link for download the installer of gcc 3.4.6 !
    I do not have a link for GCC-3.4.6 binaries for windows. I just compiled the sources tar under openSUSE. But you can download GCCs source from here and try to compile it under Cygwin or MinGW.

  5. #35
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by Christian
    Hi again!
    Sorry for the late reply. So, finally I do have some fresh linux binaries for you. I included binaries produced with GCC 3.4.6 and 4.1.2.

    CCM 1.30a (win32+linux32)
    Thanks Chris!

    Mirror: Download

  6. #36
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,583
    Thanks
    234
    Thanked 160 Times in 90 Posts
    INTEL Core Duo 2 2gb Ram

    SFC Test
    CCMX Opt.6
    10.737.064 Bytes
    comp. 36,972 sec.-
    dec. 37,812 sec.-
    CCM Opt. 6
    10.820.191 Bytes
    comp.31.701 sec.
    dec. 32.564 sec.

    ENWIK8 opt.7
    -> 20.857.925 compr. 1326 KiB/s

  7. #37
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quick test...

    Test machine: Intel PIII (Coppermine) @750 MHz, 512 MB RAM, Windows 2000 Pro SP4

    Test File: ENWIK8 (100,000,000 bytes)

    Executable: ccm.exe (model size 4)

    Timed with AcuTimer v1.2


    CCM 1.26b (Oct 29 2007), Copyright (c) 2007 C. Martelock

    Allocated 276 MiB of memory.
    97656.25 KiB -> 21565.78 KiB (ratio 22.08%, speed 258 KiB/s)

    Elapsed Time: 00:06:21.391 (381.391 Seconds)


    CCM 1.30 - Copyright (c) 2007-2008 C. Martelock - Jan 7 2008

    Allocated 274 MiB of memory.
    97656.25 KiB -> 21586.76 KiB (ratio 22.10%, speed 289 KiB/s)

    Elapsed Time: 00:05:41.642 (341.642 Seconds)


    CCM 1.30a - Copyright (c) 2007-2008 C. Martelock - Jan 9 2008

    Allocated 274 MiB of memory.
    97656.25 KiB -> 21586.76 KiB (ratio 22.10%, speed 288 KiB/s)

    Elapsed Time: 00:05:43.196 (343.196 Seconds)

  8. #38
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    630
    Thanks
    288
    Thanked 252 Times in 128 Posts
    Nice release! Would you be interested in a merge with Precomp
    (using Precomp DLL)? This would lead to a compression ratio
    like lprepaq, but speed would be better.
    http://schnaader.info
    Damn kids. They're all alike.

  9. #39
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 163 Times in 18 Posts
    Hi Christian!

    Quote Originally Posted by schnaader
    Nice release! Would you be interested in a merge with Precomp (using Precomp DLL)?
    Thanks for the kind offer. Ill contact you via mail.

  10. #40
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    These results show what we can expect from the merge.

    Precomp v0.3.7 + CCMx v1.26b (7)

  11. #41
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    630
    Thanks
    288
    Thanked 252 Times in 128 Posts
    Quote Originally Posted by LovePimple
    These results show what we can expect from the merge.

    Precomp v0.3.7 + CCMx v1.26b (7)
    Perhaps with EXE filters on the CCM side there could be an improvement like LPREPAQ made to beat paq8o8 SFC result.
    The main thing that improves comparing to LPREPAQ is the speed, Precomp+CCM will be twice as fast or even faster!

    Quote Originally Posted by Christian
    Thanks for the kind offer. Ill contact you via mail.
    BTW, have you already send a mail? Nothing in my inbox so far...
    http://schnaader.info
    Damn kids. They're all alike.

  12. #42
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by schnaader
    Perhaps with EXE filters on the CCM side there could be an improvement like LPREPAQ made to beat paq8o8 SFC result.

  13. #43
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Do you think the merge will cause it to be excluded from the SFC test?

  14. #44
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    630
    Thanks
    288
    Thanked 252 Times in 128 Posts
    Quote Originally Posted by LovePimple
    Do you think the merge will cause it to be excluded from the SFC test?
    I dont know. I recently send Werner an e-mail and asked him if
    he will add LPREPAQ 1.3 to maximumcompression, but
    he hasnt answered yet...
    Its obvious why LPREPAQ 1.1 (had a bug that caused it to crash
    sometimes) and paq8o8pre (too slow I think) werent added, but
    LPREPAQ 1.3 should be fast enough and the CCM+Precomp merge
    will be even faster, so there should be a good chance that itll be
    added.

    However, Precomp is far away from being perfect and I think there
    also could be a crash or some bug that slows it down extremely
    while processing MFC...
    So we will see...
    http://schnaader.info
    Damn kids. They're all alike.

  15. #45
    Member
    Join Date
    Feb 2008
    Posts
    4
    Thanks
    0
    Thanked 0 Times in 0 Posts
    I love this tool. I would love It even more if it did folders.

  16. #46
    Member
    Join Date
    May 2008
    Location
    England
    Posts
    325
    Thanks
    18
    Thanked 6 Times in 5 Posts
    You can always use QFC http://www.geocities.com/jadoxa/qfc/index.html or TAR to group them up before comrpessing.

  17. #47
    Member
    Join Date
    Feb 2008
    Posts
    4
    Thanks
    0
    Thanked 0 Times in 0 Posts
    I have been taring them before compression, just it gets annoying sometimes.

  18. #48
    Member
    Join Date
    May 2007
    Location
    Poland
    Posts
    91
    Thanks
    10
    Thanked 4 Times in 4 Posts
    I like TarCCM for this purpose. Once you set it up it is very convenient.
    http://www.encode.su/forums/index.ph...um=1&topic=412

  19. #49
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    926
    Thanks
    58
    Thanked 116 Times in 93 Posts
    <offtopic>

    its there a benefith to use tar instead of just 7-zip with store method ?

    7-zip 7z/store gives smalles files then 7-zip tar

  20. #50
    Member
    Join Date
    Jan 2007
    Location
    Moscow
    Posts
    239
    Thanks
    0
    Thanked 3 Times in 1 Post
    7zip gives better results archiving different file types. When files are of the same type .tar gives better results. Files are bigger because of file alignment inside of archive. 7zip compresses list of file names, tar places them as is. So if comressor is stronger then 7zip it takes benefit from raw file names placement and vice versa.

  21. #51
    Member
    Join Date
    Feb 2008
    Posts
    4
    Thanks
    0
    Thanked 0 Times in 0 Posts
    +1 to jethro for the link

  22. #52
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    926
    Thanks
    58
    Thanked 116 Times in 93 Posts
    "nimdansk

    thanx for the info.


    TAR
    no comrpession of file names
    no sroting of files


    7-zip store
    compression of filenames (hurts futher compressions)
    sorting of files by extension (helps further compression)



    Did i get it right ?

  23. #53
    Member
    Join Date
    Jun 2009
    Location
    Kraków, Poland
    Posts
    1,505
    Thanks
    26
    Thanked 136 Times in 104 Posts
    you can disable header compression in 7- zip.

  24. #54
    Member
    Join Date
    Feb 2008
    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Is there some kind of setting that you have to use to utilize both cores on a dual-core cpu. I can't get CCM to utilize more than one core and I'm only getting ~1500 KiB/s with a C2D e6700 @ 3.2ghz / 2gb ram.

    Using "ccmx 7"

  25. #55
    Member
    Join Date
    Feb 2009
    Location
    USA
    Posts
    58
    Thanks
    0
    Thanked 0 Times in 0 Posts
    CCM uses a context mixing algorithm, which is extremely difficult to do in more than one thread. So therefore, CCM does not have a multithreaded switch, and multi-core systems will only use 1 core.

    There have been quite a few interesting comments about the subject of multithreading with CCM in some of the older CCM threads.

    One exciting aspect of combining precomp with CCM however, is that the precomp work could be pushed to another thread, so those with multicore systems would not see much of a performance hit with "preCCM" versus plain CCM. The only thing they'd notice different is an out-of-this-world ratio.

  26. #56
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 163 Times in 18 Posts
    Here's a new compilation of CCM. Nothing really new, but I moved to GCC 4.3.0 using profile optimization. Thanks again to Hahobas (and of course the GCC guys) who pointed my at this technique. It might be a tiny tiny bit faster - or not. Just try it out.

    CCM 1.30b

  27. #57
    Member
    Join Date
    Feb 2009
    Location
    USA
    Posts
    58
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Christian. Thanks for your continued work on ccm. Here's some benchmarks.

    Test System: AMD Athlon 64 3000+ (Venice, single core) with 2GIG Corsair DDR CL2 RAM
    ccm130a->ccm.exe c 3 enwik8 (22351098 Bytes) - 1 m 25 s -
    ccm130a->ccm.exe d enwik8 - 1 m 24 s -
    ccm130b->ccm.exe c 3 enwik8 (22351098 Bytes) - 1 m 28 s - compression 3.5% slower
    ccm130b->ccm.exe d enwik8 - 1 m 30 s - decompression 7.1% slower

    ccm130a->ccm.exe c 7 enwik8 (21980533 Bytes) - 1 m 28 s -
    ccm130a->ccm.exe d enwik8 - 1 m 29 s -
    ccm130b->ccm.exe c 7 enwik8 (21980533 Bytes) - 1 m 31 s - compression 3.4% slower
    ccm130b->ccm.exe d enwik8 - 1 m 32 s - decompression 3.3% slower

    ccm130a->ccmx.exe c 3 enwik8 (21646059 Bytes) - 1 m 52 s -
    ccm130a->ccmx.exe d enwik8 - 1 m 53 s -
    ccm130b->ccmx.exe c 3 enwik8 (21646059 Bytes) - 2 m 3 s - compression 9.8% slower
    ccm130b->ccmx.exe d enwik8 - 1 m 58 s - decompression 4.4% slower

    ccm130a->ccmx.exe c 7 enwik8 (20857925 Bytes) - 1 m 56 s -
    ccm130a->ccmx.exe d enwik8 - 1 m 58 s -
    ccm130b->ccmx.exe c 7 enwik8 (20857925 Bytes) - 2 m 1 s - compression 4.3% slower
    ccm130b->ccmx.exe d enwik8 - 2 m 2 s - decompression 3.3% slower

    Maybe the intel folks will see an improvement with 130b.

  28. #58
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 163 Times in 18 Posts
    Hmmm, that's not good. On my system CCM and CCMx was around ~1% faster for compression and decompression. But on the other hand, the profile feedback was generated on MY box using much more data than just ENWIK.
    Anyway, I think I'll continue to use profile optimization in the future. It's much more convenient to use it than hand tuning the source - e.g. for Slug it really helped (no more trial and error coding). RZM's decompression is faster, too. And I still hope that GCC's automagical optimizations will improve further.
    Prefetching is another story, but I haven't tried it out yet.

  29. #59
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Thanks Chris!

    The latest CCM and RZM are now available for download from my site.

    http://www.geocities.com/lovepimple_mail/

    I have been busy today or they would have been available much sooner.

  30. #60
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 163 Times in 18 Posts
    Here's a new compile. I removed some compiler switches and changed the data set on which the profile feedback was created. On my C2D system these executables are a tiny bit faster for compression and a little faster for decompression (than 1.30a). Now, I'll wait for some benchmarks to see if the compiler settings are heading in the right direction.

    Hahobas:
    Can you rerun your tests just to be sure that the timings are right. Maybe you can check something different than enwik, too?

    Download CCM 1.30c

Page 2 of 3 FirstFirst 123 LastLast

Similar Threads

  1. CCM decompiling
    By Shelwien in forum Data Compression
    Replies: 13
    Last Post: 17th August 2009, 21:47
  2. CCM 1.25 is here!
    By Christian in forum Forum Archive
    Replies: 84
    Last Post: 16th November 2007, 11:00
  3. CCM(x) multithreaded ?
    By SvenBent in forum Forum Archive
    Replies: 2
    Last Post: 15th September 2007, 16:29
  4. CCM 1.2x branch
    By Christian in forum Forum Archive
    Replies: 107
    Last Post: 8th June 2007, 18:56
  5. CCM - 1.1.x branch
    By Christian in forum Forum Archive
    Replies: 105
    Last Post: 20th March 2007, 00:50

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •