Results 1 to 27 of 27

Thread: Freeware TarCCM Archiver

  1. #1
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    I have written an archiver that utilises the power of Chris Martelock's awesome CCM file compressor. The archiver works in the same way as my freeware "Send To" interface; but with the input being tarred before compression.

    Download TarCCM Archiver

  2. #2
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    Thank you!

  3. #3
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    Thanks LovePimple! Mirrored

  4. #4
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    Thanks LovePimple! Doublemirrored

    TarCCM Archiver v1.0


  5. #5
    Member
    Join Date
    Apr 2007
    Posts
    12
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Really nice tool. Thanks !!
    I just have a question : wouldn't it be better if it used 7-Zip with zero-compression instead of Tar ? It may give better results since 7-zip groups the files with the same extension before compressing.

  6. #6
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by Aidanael
    I just have a question : wouldnt it be better if it used 7-Zip with zero-compression instead of Tar ?
    Possibly! Perhaps someone could test this theory?

  7. #7
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    Originally I thought that TARring should be faster than 7z format... but I have no arguments for proving it

  8. #8
    Member
    Join Date
    Apr 2007
    Posts
    12
    Thanks
    0
    Thanked 0 Times in 0 Posts
    TARing is much faster than 7-zip with zero-compression, but the resulting archive is not solid, apparently, so it is bigger.
    In terms of speed, tar.ccm is better, but in terms of performance, I think 7z.ccm should be better.

  9. #9
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    to be exact, freearc would be much better. unlike 7z, it sorts files in an order defined in arc.groups file (it's like rarfiles.lst), moves similar files together (it's a feature unique among archivers) and sorts small files by path while large files are sorted by size (seems that the same algorithm used in uharc). also it has 2-3 times less memory reqs, uses separate thread to precache data archived, so on, so on recommended cmdline for you:

    arc a archive -r -m0 -dm0 -ds=gercpn --cache=64mb

    (for regular compression "gercpn" order used by default, but for -m0 compression it's disabled because it can't increase compression ratio

    www.haskell.org/bz

  10. #10
    Tester

    Join Date
    May 2008
    Location
    St-Petersburg, Russia
    Posts
    182
    Thanks
    3
    Thanked 0 Times in 0 Posts
    Bulat, what about next version of your great FreeArc (with TTA for bmp and wav)? It will be very well for most SFC charts...

  11. #11
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    I made a little test - compressing Dragon History, an old Czech adventure game (some info on http://www.ucw.cz/draci-historie/index-en.html)

    Starting filesize: 154,715,960B

    Name of archiver__Archiving time__Resulting size after CCM 1.20g 4
    7-zip____7s__46,637,252
    winrar___8s__46,578,425
    freearc__9s__46,643,150
    tar_____11s__46,577,963

    Seems funny...

  12. #12
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    definitely

    try to compress gimp sources, and look at arc with and without -ds switch. former should give the best compression while latter should be faster. the result depends on the contents of rarfiles.lst/arc.groups, though

  13. #13
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    Quote Originally Posted by Squxe
    Bulat, what about next version of your great FreeArc (with TTA for bmp and wav)? It will be very well for most SFC charts...
    there are 2 problems that i still need to fix. also, i want to finish tornado and inlcude it here, and implement features requested by Awe. the programs developed for people, not for charts

    although, i still dont understand why arc is worser than 7z in your tests. if you dont want to make your files public available, we can check on something else. in particular, dict+lzp+ppmd should have much better compression of text than lzma

  14. #14
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    Black_Fox

    are you cleared cache before each test? if not, you can't see benefits of pre-caching data

    ... otoh, data caching anyway meaningless if you can't compress simultaneously with data reading. the fastest solution will be any program that puts its result to stdout (tar, rar, 7z) and CCM that compress data from stdin

    i'm pretty sure that your results are explained by lack of groups sorting in tar and rar (freearc, like rar, disables file sorting in -m0 mode, but i suggested you to enable it againby -ds). in most cases, file sorting improves compression, but it seems that not in your case. probably, without -ds switch freearc will slightly outperform both rar and tar, bit it's better to change your testset to one where file sorting shows compression improvements

  15. #15
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    No, I havent cleared cache.
    I used this commandline:
    Quote Originally Posted by Bulat Ziganshin
    arc a archive -r -m0 -dm0 -ds=gercpn --cache=64mb
    Anyway, Ill try to compress GIMP.

  16. #16
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    GIMP 2.2.9 sources (82,733,191) archived and then compressed by ccm 1.20g

    archiver / size
    arc* / 8,367,896
    rar / 8,444,707
    tar / 8,447,399
    arc / 8,480,945
    7z / 8,609,526

    again used the same commandline, the asterisked (*) arc means without -ds swtich

    used 7-zip 4.45beta, WinRAR 3.70beta5, FreeArc 0.36

    EDIT: Changed ratios into sizes (was very stupid mistake indeed)

  17. #17
    Member
    Join Date
    Apr 2007
    Posts
    12
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Weird, indeed. Did you use 7-zip, freearc, and WinRAR to STORE data instead of compressing it ?
    I gave a try on my Mugen directory (~1380 Mb), reduced to ~699 Mb with 7z.ccm, and ~708 Mb with tar.ccm.

    Anyway, I'll give a try on GIMP sources too, just to check.

  18. #18
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    you can disable sorting and directory compression in 7z too, it will give results close to arc*

    but anyway, file sorting should improve compression (it's why it was invented, after all) and both you? results are rather strange. on my own tests which includes thousands of files, sorting significantly (5-10%) improves overall compression ratio

  19. #19
    Member
    Join Date
    Apr 2007
    Posts
    12
    Thanks
    0
    Thanked 0 Times in 0 Posts
    GIMP 2.2.9 sources (82.733.191 B)

    using CCMx c 5 (from best to worst comp ratio)
    -> gimp-2.2.9.arc (82 876 668 B) -> gimp-2.2.9.arc.ccm (7 958 192 B)
    -> gimp-2.2.9.tar (86 392 832 B) -> gimp-2.2.9.tar.ccm (7 985 481 B)
    -> gimp-2.2.9.rar (83 094 766 B) -> gimp-2.2.9.rar.ccm (8 008 424 B)
    -> gimp-2.2.9.7z (82 790 190 B) -> gimp-2.2.9.7z.ccm (8 064 782 B)

  20. #20
    Member
    Join Date
    Apr 2007
    Posts
    12
    Thanks
    0
    Thanked 0 Times in 0 Posts
    My MSYS directory (352 896 652 B)

    using CCMx c 5 (from best to worst comp ratio)
    > MSYS.arc (353 159 032 B) > MSYS.arc.ccm (37 666 443 B)
    > MSYS.7z (352 995 103 B) > MSYS.7z.ccm (38 209 581 B)
    > MSYS.tar (360 401 408 B) > MSYS.tar.ccm (38 534 425 B)


    Blender 2.43 sources (59 334 447 B)

    using CCMx c 5 (from best to worst comp ratio)
    > blender-2.43.arc (59 474 386 B) > blender-2.43.arc.ccm (8 087 834 B)
    > blender-2.43.tar (62 914 560 B) > blender-2.43.tar.ccm (8 119 177 B)
    > blender-2.43.7z (59 396 424 B) > blender-2.43.7z.ccm (8 172 539 B)

    Thank you for making me discover FreeArc

  21. #21
    Member
    Join Date
    Jun 2009
    Location
    Kraków, Poland
    Posts
    1,474
    Thanks
    26
    Thanked 121 Times in 95 Posts
    7zip by default compresses file information (ie. headers) with lzma. so this can be the reason for lower performance.

    try disabling header compression in 7z format (option removed from newer 7zip versions).

  22. #22
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    its definitely the reason. btw, in my program, vice versa, any algorithm may be used for directory compression

    Quote Originally Posted by Aidanael
    Thank you for making me discover FreeArc
    good results is consequence of freearc flexibility. anyway, i can find options combinartion which allows to mimic anything, from arj to winrk

    and by popular requests, i plan to add ccm compression in next version, as i already fone with ppmonstr compression - by calling external ccm.exe

    btw, if you have ppmostr.exe in path - try to compress sources with -m6p instead of -m0. i hope that it will be faster/better than using tar+ccm

  23. #23
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 153 Times in 18 Posts
    Thank you for your work on TarCCM, LovePimple. Good job!

  24. #24
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Thank you for the kind words, Chris!

  25. #25
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    I have updated TarCCM and the CCM STI to make them compatible with CCM v1.26b.

  26. #26
    Member Vacon's Avatar
    Join Date
    May 2008
    Location
    Germany
    Posts
    523
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Hello everyone,

    thank you LovePimple!

    Best regards!

  27. #27
    Member
    Join Date
    Feb 2008
    Posts
    4
    Thanks
    0
    Thanked 0 Times in 0 Posts
    I couldnt get TarCCM or CCM STI to work. Both said, Could not find "ccm.exe" in path. I have move ccm around the directory and couldn't get it to work, where should I have it placed?

    EDIT:Woops, sorry. I see it says anything in windows folder.

Similar Threads

  1. RTIME - New freeware program
    By LovePimple in forum The Off-Topic Lounge
    Replies: 3
    Last Post: 4th July 2008, 11:47
  2. XTIME - New freeware program
    By LovePimple in forum The Off-Topic Lounge
    Replies: 4
    Last Post: 4th July 2008, 04:40
  3. Freeware "Send To" interface for CCM and QUAD
    By LovePimple in forum Forum Archive
    Replies: 2
    Last Post: 20th March 2007, 18:22

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •