FreeArc

[jethro]

OK, thanks that will do it.
But I'm curious how does Rar achieves that? Because Rar exe can be run from this 3.99GB file on my computer? So if this is possible in Rar, it should be possible in FreeArc, no?

Also I have a suggestion on executing files from GUI and if unpack only 1 file or everything.
Simply, ask the user something like:

'Do you want to run this file only or unpack archive first?'
[Run] [Unpack] [Cancel]

Rar unpacks for exe and html but sometimes i know its not necessary or I dont want to wait so I think a choice for the user is a good solution.

[m^2]

I'm not 100% certain (and won't check it now), but I don't think Windows maps overlay data.
So the guessed cause seems incorrect to me.

[Piotr Tarsa]

From what I remember, WinRAR just concatenates SFX module and archive data and Windows doesn't load that excessive data. Instead SFX module opens file in which it's contained and search the first few megabytes or so for RAR archive signature. I think so. When you try to open some file in WinRAR it firstly scans it (remember, only the first few megabytes or so) for archive signatures and when it finds some then it doesn't fire shell commands to open but opens the archive inside.

You can try it yourself. Concatenate small text file and ZIP file. WinRAR will succesfully detect the ZIP archive and open it (treating the concatenated text + ZIP as ZIP SFX). But if you concatenate a very big PDF and then some ZIP and then another big PDF then WinRAR probably won't find the ZIP signature.

Things may have changed since I've checked this, but probably the above statements are still true.

[Intrinsic]

FreeArc hits the mainstream press:
http://www.tomshardware.com/reviews/...sion,2572.html

Shame they didn't have someone who knew what they were doing and could full exploit it's awesomeness to do the tests(for all the compressors), they should have also linked in popular compression benchmark sites like maximum etc.

[JangoFatXL]

like maximum etc.

I told them about maximumcompression when I suggested to report about freearc 0.60 (in december I think)...

[Mexxi]

What a strange article. At first they give a nice overview over the contestants as if they wanted to introduce them to those who don't know them. Then they explain the benefits in real-life working environments and the therefore necessary features. All nice and understandable. So they cater to the everyday user. Why do they use command line tests then?? That doesn't make any sense. They should have compared the efficiency of the GUI-settings, because that's what matters in an office environment.

As for a command line test, this was a joke. They use a comp with 8GB RAm and don't even specify the dictionary size for 7z. On what grounds can they compare these programs if they don't even use settings that max out their performance? And because of that, their little compression time+compression size calculation is totally moot.

[Mexxi]

I have an idea for a new feature for Freearc. It should support sort-lists, that contain the order of all files that are about to be compressed. This would enable the user to control sorting on a file by file basis and not just in groups. It would be a nice enhancement to the sorting options it already has.

If Freearc would support absolute paths in sort-lists, then this would also enable users to compress content that is spanned across different directories or even drives.

[IsName]

I have an idea for a new feature for Freearc. It should support sort-lists, that contain the order of all files that are about to be compressed. This would enable the user to control sorting on a file by file basis and not just in groups. It would be a nice enhancement to the sorting options it already has.

If Freearc would support absolute paths in sort-lists, then this would also enable users to compress content that is spanned across different directories or even drives.

Already implemented. Just call freearc with -ds -s --append parameters for every solid block.

[Mexxi]

Thanks for the hint, but that's not the same. Everything should be in one solid block. Also, using append requires the user to use a new command line for every file-type or even file. Consider doing that dozens of times in a row for some excessive archives plus the possibility to screw up filenames with typos. A surefire road to frustration. It's much faster creating a sort list that specifies the exact archive layout and that also gives you the benefit of stashing everything in a single solid block.

[IsName]

Thanks for the hint, but that's not the same. Everything should be in one solid block.

Be careful! The entire solid block uses the same chain of compression algos.

Also, using append requires the user to use a new command line for every file-type or even file. Consider doing that dozens of times in a row for some excessive archives plus the possibility to screw up filenames with typos. A surefire road to frustration.

Use a script. As much work as possible should be done by shell, not by program.

It's much faster creating a sort list that specifies the exact archive layout ...

I think that -ds does suppress file sorting and FA will put files into the archive exactly in specified order.

[Bulat Ziganshin]

I have an idea for a new feature for Freearc. It should support sort-lists, that contain the order of all files that are about to be compressed

i plan to support it other way: freearc should compress files in the order given on cmdline/listfiles if -ds was used if there were no wildcards nor -r

I think that -ds does suppress file sorting and FA will put files into the archive exactly in specified order.

unfortunately, no. now it reads list of files in the directory and selects files those names match wildcards given on cmdline. effectively this sorts files by name, on NTFS at least

[Mexxi]

Be careful! The entire solid block uses the same chain of compression algos.

That's my intention

Use a script. As much work as possible should be done by shell, not by program.

That adds unnecessary overhead and I think so does the constant updating. Imagine archiving tens of thousands of files. The updating routine alone would slow things down more than necessary. Freearc has to create a list of files internally anyway, so the work you speak of is being done anyway, just not as versatile as it could be.

So far I have been using sorted isos to achieve the same effect. It takes about 2-5 minutes to create a sortfile for those and it saves you the overhead a script plus the constant archive updating would cause, let alone the extra i/o. Still, this adds an extra container to the mix and is not really necessary.

I think that -ds does suppress file sorting and FA will put files into the archive exactly in specified order.

Yes, but only defined by groups, or as I understand it: by file endings. This is not the same as being able to specify the sorting of single files. Example: Currently I'm testing compression on Chronicles of Riddick - Escape from Butcher's Bay. The game uses a single container format for almost all file types, meaning easy to compress scripts as well as hard to compress audio files. So using groups is not effective here, because the containers all use the same file endings.

i plan to support it other way: freearc should compress files in the order given on cmdline/listfiles if -ds was used if there were no wildcards nor -r

Thanks for the answer Any kind of support would be cool and the way you plan to do it sounds good to me. While I'm at it, think about also implementing it into the gui-version. People could drag and drop folders and files into a specified list area and then use drag and drop again to define their sorting. This would be a first in the archiving world and certainly a not too hard to implement feature

Disktrix Ultimate Defrag uses a similar system in their gui to define the placement of files during defragmentation. I find it quite great.

[Mexxi]

I wanted to test srep 1.42 on my precomped vm.vmdk-file. It's of course srepped and stored in an arc-archive. Opening the arc-file only gives me the option to extract the original vm.vmdk, but neither the precomped container nor the srepped one. It would be great if Freearc would allow such a parsing of embedded containers.

That being said, I tried to extract the srep-file manually by cutting off the header and the footer of the arc-file, but srep complained that the checksum of the compressed and decompressed file wouldn't match. What did I do wrong?

Eit: I found a workaround by grabbing the extracted file directly off of Freearc's temp-folder when extracting vm.vdmk. Still, I wonder why my editing attempt failed...

[Bulat Ziganshin]

new version:

• Added dispack filter for executable files (also available in unarc/sfx/unarc.dll)
• Fixed bug in -mex2 mode (i.e. 4x4:tor) detected on "Oblivion - Sounds.bsa"
• -ioff/--shutdown option for powering off computer after completion of operation
• GUI: shutdown option in Add/Extract dialogs
• GUI: storing expanded status of options block inside Progress window
• GUI: disable unused buttons in Progress window after operation was finished

The most exciting thing is new dispack filter. These are results of compression of MS Office installation joined into single 810 mb file with various filters + lzma:max:

no filter: 323,430,022 bytes
bcj:       315,808,982
bcj2:      311,737,777
dispack:   311,536,260

I hope that compression will be increased in next versions by improving x86 code detection algorithm. Please note that dispack compression format will probably be chnaged too, so don't expect that archives made by this version can be extracted by future ones. nevertheless, dispack is ready for tests and making installers, making FreeArc executables compression comparable with 7zip one

[zody]

Hi Bulat,
it's nice to see the new version of FreeARC.
Thanks for adding the shutdown-feature.
Dispack seems to achieve really good compression ratio...I've read something about disoack quite a long time ago; I forgot that dispack should be used in FreeARC...
Is dispack used as default filter ?
If it helps to improve the filter, I'll do some tests with bcj and dispack

[Bulat Ziganshin]

dispack now is highly experimental feature so it not used by default and cannot be used for a long time. nevertheless, it may be interesting for those who make installers

[Skymmer]

Let's see how dispack performs. I used 3 different testsets.
win.bins is the collection of 2007 .exe and .dll files from dllcache folder of Win XP Pro SP2 Rus. Original size is 373 964 014 bytes.
my.pack is my own collection of PE files used to test PE packers. 29 files with size of 391 616 000 bytes.
ins.shield is the collection of all .exe, .dll and .ocx files from InstallShield 2010 Express install. 521 files with size of 303 727 350 bytes.
For both FreeARC and 7-zip lzma:a1:d128m:lc3:fb64:mfbt4 used as compression engine.

                  win.bins        my.pack      ins.shield
                -----------     ----------     ----------
7z              113 205 444     85 115 115     61 391 579
7z + BCJ        110 012 814     78 805 909     59 215 166
7z + BCJ2       106 690 622     77 061 283     56 767 111
FA              113 476 317     85 115 114     61 457 775
FA + EXE        109 970 344     78 805 901     59 034 340
FA + Dispack    107 992 995     75 755 245     56 396 215

Not bad! Thank you man

EDIT: Just thought about one idea. Need to check.

[Bulat Ziganshin]

Downloads in Nov'09..Feb'10: about 10k/month
Downloads in Mar'10: 25k

it's result of paper inspired by Jango. not bad


to Skymmer: of course, dispack by itself is much better than bcj2. i just need to make good code/data detector

[Cyan]

Indeed the beginning of well-deserved notoriety ...

[Stephan Busch]

Hi there.

I'm going to release new benchmarks tomorrow (also including FreeArc 0.61 from february).

How can I enable dispack in newest FreeARC 0.61? Is there something in the arc.ini that must be changed for the GUI version or is it a switch in the cmd line version?

[Bulat Ziganshin]

it's easy just dump settings from program (arc.exe --print-config >arc1.ini) then s/exe/dispack/ and insert these settings into your arc.ini

OTOH, dispack is very experimental now so you will somewhat fool your readers

btw, it would be very interesting to see results of -mex1...5 modes, since they are going to be standrad in next FA version

[JangoFatXL]

hi bulat, none of the links on freearc.org to encode.su forums work anymore since the hoster was changed to dreamhosters.com...

[Bulat Ziganshin]

thank you, i have fixed as much as possible. unfortunately it seems that we lost old forum archives ((

[Shelwien]

Its mirrored in google groups.
Also I have a local copy in .msg format if you want.

[Bulat Ziganshin]

new version. main change is that m/t compression is now used by default:

• -m1..-m4 compression methods are equal to -mex1..4 if you run FreeArc on multicore cpu

This change required to fix usability quirks reported by users, so now:

• threads performing compression/decompression/encryption tasks are assigned lower priority so computer no more hangs when you run compression
• progress indicator was made smoother, especially for m/t compression
• compression settings in 4x4 are automatically reduced to obey memory limit settings
• 4x4's :t and :i params are no more saved in archive

Also, syntax for describing compression method in -m option/arc.ini/built-in was extended with "|" syntax:

• -m=tor|lzma|lzma:max means "use tor for 1-core cpu, lzma for 2 cores, lzma:max for 3+ cores
• -m=tor|||lzma means tor for 1-3 cores, lzma for 4+ cores (i.e. repeated strings may be omitted)
• -m=rep+(tor|lzma) means rep+tor for 1 core, rep+lzma for 2+ cores (i.e. use parens to alter only part of line depending on number of cores)
• -m=(|x)tor mean tor for 1 core, xtor for 2+ cores (i.e. it's just text replacement so you can do it on any part of line)
• use "arc --print-config" to see how new syntax used to define -m1..-m4 methods
• don't forget to use quotes around options containing "|" in cmdline! example:

  arc a archive "-m=lzma:(fast|normal)"

dispack:

• option :b allows to control block size: dispack:b8m
• default block size is 64mb
• memory usage: compression - 3.25*blocksize, decompression - 2.25*blocksize,

other changes:

• zip/rar/7z/... archives now opens in FreeArc when double-clicked in Explorer
• disabled "New archive" Explorer command since we don't yet support Drag&Drop
• when GUI operation finished, uiMessage==" " so that this label no more shrinked
• fixed bug: Tornado compression allocated 2.5*dict memory, now it's 1.5*dict
• new ukrainian translation by ruduk

Finally, FreeArc provides 100% cpu utilization in fast modes (-m1..4). If you need maximum m/t compression (like 7zip -mx9 -m=lzma2), -mex5..9 should be used, that are bit worser than -m5..9.

One drawback of new scheme is that the same compression mode (say, -m4) on m/t cpu becomes much faster but provides even less compression that on single-core - this looks rather awkward, so i may need to rebalance entire methods line

[JangoFatXL]

zip/rar/7z/... archives now opens in FreeArc when double-clicked in Explorer

Maybe it would be better to let the user decide whether he or she wants by default FreeArc to open foreign archives.
A checkbox in the installer and in the settings section would be useful, I think
Let's say in next public release or something like that.
The reason is quite simple: most of the people (inluding me ) do not like a program "stealing" linkage to a extension without asking before...

[giorgiotani]

I've been reported by a PeaZip user that self extracting archives created in FreeArc format (PeaZip uses command line executable arc.exe, all files are from FreeArc 0.60 release) fails on extraction if encryption was used, reporting "unsupported compression" error message
http://sourceforge.net/tracker/index...96&atid=887372

In example the error happens when I launch extraction of an sfx archive that way:
arc.exe a -m4 -s -rr -sfxfreearc.sfx -paaa -ae=aes "nameofthearchive" "nameoftheinput"

The same error happens as well as I try different compression and encryption methods and with different sfx modules, also changing the order of the various switches.
Btw, the self extracting archive reports the error without asking for password, so it seems it does not detect the associated data is encrypted befor starting the extraction.

If I rename the resulting archive.exe in archive.arc, it can be opened and extracted without problems, meaning that the error the self-extraction reports is a false positive as all the data is ok.
Moreover, if I remove the "-sfxfreearc.sfx" part, arc.exe create a perfectly valid .arc file, so I think the problem is in the sfx modules and not in the command definition, as the previous tests with various compression/extraction parameters have hinted.

I apologize in advance if this problem was already reported, I was not able to find it on the application's forums and tracker.

[Bulat Ziganshin]

thank you for report. it's already on tracker: http://code.google.com/p/freearc/issues/detail?id=77 - searching by "sfx" or "encryption" should find it. in freearc, encryption methods is just a subset of compression methods

[giorgiotani]

Thank you for the answer, I'm glad to see the issue is already tracked and that the fix is scheduled in milestone 0.70.

[Bulat Ziganshin]

new version:

• Settings: separate checkbox for associating FreeArc with zip/rar/... archives
• Compression: stronger methods for -mex3t/-mex4t
• GUI: show full current directory name in window title when outside of archive
• fixed: don't use more memory than set in -lc (reported by Sami)
• fixed: bug on computers with memory > 4gb (it was treated as -lc- option)
• fixed: arc l archive_with_rr
• i18n: two more strings