Freeware TarCCM Archiver

[LovePimple]

I have written an archiver that utilises the power of Chris Martelock's awesome CCM file compressor. The archiver works in the same way as my freeware "Send To" interface; but with the input being tarred before compression.

Download TarCCM Archiver

[encode]

Thank you!

[Black_Fox1]

Thanks LovePimple! Mirrored

[encode]

Thanks LovePimple! Doublemirrored

TarCCM Archiver v1.0

[Aidanael]

Really nice tool. Thanks !!
I just have a question : wouldn't it be better if it used 7-Zip with zero-compression instead of Tar ? It may give better results since 7-zip groups the files with the same extension before compressing.

[LovePimple]

I just have a question : wouldnt it be better if it used 7-Zip with zero-compression instead of Tar ?

Possibly! Perhaps someone could test this theory?

[Black_Fox1]

Originally I thought that TARring should be faster than 7z format... but I have no arguments for proving it

[Aidanael]

TARing is much faster than 7-zip with zero-compression, but the resulting archive is not solid, apparently, so it is bigger.
In terms of speed, tar.ccm is better, but in terms of performance, I think 7z.ccm should be better.

[Bulat Ziganshin]

to be exact, freearc would be much better. unlike 7z, it sorts files in an order defined in arc.groups file (it's like rarfiles.lst), moves similar files together (it's a feature unique among archivers) and sorts small files by path while large files are sorted by size (seems that the same algorithm used in uharc). also it has 2-3 times less memory reqs, uses separate thread to precache data archived, so on, so on recommended cmdline for you:

arc a archive -r -m0 -dm0 -ds=gercpn --cache=64mb

(for regular compression "gercpn" order used by default, but for -m0 compression it's disabled because it can't increase compression ratio

www.haskell.org/bz

[squxe]

Bulat, what about next version of your great FreeArc (with TTA for bmp and wav)? It will be very well for most SFC charts...

[Black_Fox1]

I made a little test - compressing Dragon History, an old Czech adventure game (some info on http://www.ucw.cz/draci-historie/index-en.html)

Starting filesize: 154,715,960B

Name of archiver__Archiving time__Resulting size after CCM 1.20g 4
7-zip____7s__46,637,252
winrar___8s__46,578,425
freearc__9s__46,643,150
tar_____11s__46,577,963

Seems funny...

[Bulat Ziganshin]

definitely

try to compress gimp sources, and look at arc with and without -ds switch. former should give the best compression while latter should be faster. the result depends on the contents of rarfiles.lst/arc.groups, though

[Bulat Ziganshin]

Bulat, what about next version of your great FreeArc (with TTA for bmp and wav)? It will be very well for most SFC charts...

there are 2 problems that i still need to fix. also, i want to finish tornado and inlcude it here, and implement features requested by Awe. the programs developed for people, not for charts

although, i still dont understand why arc is worser than 7z in your tests. if you dont want to make your files public available, we can check on something else. in particular, dict+lzp+ppmd should have much better compression of text than lzma

[Bulat Ziganshin]

Black_Fox

are you cleared cache before each test? if not, you can't see benefits of pre-caching data

... otoh, data caching anyway meaningless if you can't compress simultaneously with data reading. the fastest solution will be any program that puts its result to stdout (tar, rar, 7z) and CCM that compress data from stdin

i'm pretty sure that your results are explained by lack of groups sorting in tar and rar (freearc, like rar, disables file sorting in -m0 mode, but i suggested you to enable it againby -ds). in most cases, file sorting improves compression, but it seems that not in your case. probably, without -ds switch freearc will slightly outperform both rar and tar, bit it's better to change your testset to one where file sorting shows compression improvements

[Black_Fox1]

No, I havent cleared cache.
I used this commandline:

arc a archive -r -m0 -dm0 -ds=gercpn --cache=64mb

Anyway, Ill try to compress GIMP.

[Black_Fox1]

GIMP 2.2.9 sources (82,733,191) archived and then compressed by ccm 1.20g

archiver / size
arc* / 8,367,896
rar / 8,444,707
tar / 8,447,399
arc / 8,480,945
7z / 8,609,526

again used the same commandline, the asterisked (*) arc means without -ds swtich

used 7-zip 4.45beta, WinRAR 3.70beta5, FreeArc 0.36

EDIT: Changed ratios into sizes (was very stupid mistake indeed)

[Aidanael]

Weird, indeed. Did you use 7-zip, freearc, and WinRAR to STORE data instead of compressing it ?
I gave a try on my Mugen directory (~1380 Mb), reduced to ~699 Mb with 7z.ccm, and ~708 Mb with tar.ccm.

Anyway, I'll give a try on GIMP sources too, just to check.

[Bulat Ziganshin]

you can disable sorting and directory compression in 7z too, it will give results close to arc*

but anyway, file sorting should improve compression (it's why it was invented, after all) and both you? results are rather strange. on my own tests which includes thousands of files, sorting significantly (5-10%) improves overall compression ratio

[Aidanael]

GIMP 2.2.9 sources (82.733.191 B)

using CCMx c 5 (from best to worst comp ratio)
-> gimp-2.2.9.arc (82 876 668 B) -> gimp-2.2.9.arc.ccm (7 958 192 B)
-> gimp-2.2.9.tar (86 392 832 B) -> gimp-2.2.9.tar.ccm (7 985 481 B)
-> gimp-2.2.9.rar (83 094 766 B) -> gimp-2.2.9.rar.ccm (8 008 424 B)
-> gimp-2.2.9.7z (82 790 190 B) -> gimp-2.2.9.7z.ccm (8 064 782 B)

[Aidanael]

My MSYS directory (352 896 652 B)

using CCMx c 5 (from best to worst comp ratio)
> MSYS.arc (353 159 032 B) > MSYS.arc.ccm (37 666 443 B)
> MSYS.7z (352 995 103 B) > MSYS.7z.ccm (38 209 581 B)
> MSYS.tar (360 401 408 B) > MSYS.tar.ccm (38 534 425 B)


Blender 2.43 sources (59 334 447 B)

using CCMx c 5 (from best to worst comp ratio)
> blender-2.43.arc (59 474 386 B) > blender-2.43.arc.ccm (8 087 834 B)
> blender-2.43.tar (62 914 560 B) > blender-2.43.tar.ccm (8 119 177 B)
> blender-2.43.7z (59 396 424 B) > blender-2.43.7z.ccm (8 172 539 B)

Thank you for making me discover FreeArc

[Piotr Tarsa]

7zip by default compresses file information (ie. headers) with lzma. so this can be the reason for lower performance.

try disabling header compression in 7z format (option removed from newer 7zip versions).

[Bulat Ziganshin]

its definitely the reason. btw, in my program, vice versa, any algorithm may be used for directory compression

Thank you for making me discover FreeArc

good results is consequence of freearc flexibility. anyway, i can find options combinartion which allows to mimic anything, from arj to winrk

and by popular requests, i plan to add ccm compression in next version, as i already fone with ppmonstr compression - by calling external ccm.exe

btw, if you have ppmostr.exe in path - try to compress sources with -m6p instead of -m0. i hope that it will be faster/better than using tar+ccm

[Christian]

Thank you for your work on TarCCM, LovePimple. Good job!

[LovePimple]

Thank you for the kind words, Chris!

[LovePimple]

I have updated TarCCM and the CCM STI to make them compatible with CCM v1.26b.

[Vacon]

Hello everyone,

thank you LovePimple!

Best regards!

[Asheboy]

I couldnt get TarCCM or CCM STI to work. Both said, Could not find "ccm.exe" in path. I have move ccm around the directory and couldn't get it to work, where should I have it placed?

EDIT:Woops, sorry. I see it says anything in windows folder.