Standard for compression libraries API

[Bulat Ziganshin]

other lzss algos include quicklz, lzo, some encode's program and so on

tornado is rather universal lz77 compressor and i don't optimized it specifically for fast quicklz-like mode or lzss encoding in general. nevertheless, you can try with -11 -c1 (tor:11:c1) which has good compresssion on l;arge files just due to use of ;large dictionary and deep search

[m^2]

other lzss algos include quicklz, lzo, some encode's program and so on

tornado is rather universal lz77 compressor and i don't optimized it specifically for fast quicklz-like mode or lzss encoding in general. nevertheless, you can try with -11 -c1 (tor:11:c1) which has good compresssion on l;arge files just due to use of ;large dictionary and deep search

It's very good and -12 even slightly better.
I don't love LZSS in ex mode anymore.
Can I suggest making this one of the standard modes? Great for everything that has to be decompressed at runtime.

[Bulat Ziganshin]

what you mean by standard mode?

[m^2]

what you mean by standard mode?

On my test set FreeArc -m1x generates a 127.4 MB file (uncompressed is 248.9) and unpacks it in 6.375s.
Tornado -12 -c1: 116.1 MB and 2.250s.
It's clearly better.
I think that FreeArc could use something like -m?xx modes devoted solely to decompression speed, where different modes could compress equally slow, but would differ is decompression speed (and size, of course).
Useful for mentioned already runtime unpackers and installers.

[Bulat Ziganshin]

try also with -12 -c2/-c3/-c4 and arc -m9x

idea about line of compression modes odered by decompression speed looks unordinary. you can add it yourself into arc.ini:

[Compression methods]
1xx=tor:12:c1
2xx=tor:12:c2
3xx=tor:12:c3 / dict+tor:12:c3
4xx=tor:12:c4 / dict+tor:12:c4
5xx=9x

[pat357]

try also with -12 -c2/-c3/-c4 and arc -m9x

idea about line of compression modes odered by decompression speed looks unordinary. you can add it yourself into arc.ini:

[Compression methods]
1xx=tor:12:c1
2xx=tor:12:c2
3xx=tor:12:c3 / dict+tor:12:c3
4xx=tor:12:c4 / dict+tor:12:c4
5xx=9x

Doesn't seem to work .... are tor:c1/tor:c2 implemented in ARC or do we need to configure the "full tornado" as external compressor ?

G:\test\fp.log>arc a -mtor:12:c1 tst fp.log
FreeArc 0.50 alpha (October 11 200 started: 0.00 secs
Compressing 1 file of 20.617.071 bytes: 0.01 secs
Using tor:12:20mb:h128mb:c1
Memory for compression 156mb, decompression
Compressing fp.log 0%
ERROR: Invalid compression method or parameters in tor:12:20mb:h128mb:c1

[m^2]

Doesn't seem to work .... are tor:c1/tor:c2 implemented in ARC or do we need to configure the "full tornado" as external compressor ?

Yes, it has to be external. 4x4 also doesn't accept these modes.

try also with -12 -c2/-c3/-c4 and arc -m9x

idea about line of compression modes odered by decompression speed looks unordinary. you can add it yourself into arc.ini:

[Compression methods]
1xx=tor:12:c1
2xx=tor:12:c2
3xx=tor:12:c3 / dict+tor:12:c3
4xx=tor:12:c4 / dict+tor:12:c4
5xx=9x

If we're talking about archiving and FreeArc, decompression speed is probably rarely very important, but if FreeArcLib is meant to be a general purpose compression library, you should consider other uses too.
Nowadays when CPU power is cheap and fast IO - costly, compression is a very viable solution meant not to reduce size but rather - improve performance. This is quite often used by programs meant to run from a DVD / network shares / USB drives, but from HDDs too, i.e. games archives (ok, making access linear is more important than compression here). Simple calculation: Tornado gets over 100 MB/s on my PC. On a moderately fast new computer, using 4 cores, it would have at least 500 MB/s. That's about 6 times faster than average HDD read speed, so even with 20% size reduction you still get faster startup.
Another possible use - multimedia codec that does some preprocessing then applies FA. Here decompression speed is much more important than compression, you rarely recompress multimedia files.
"All for decompression speed" might be too extreme approach for most uses, but I think that decompression speed deserves more attention.

[lunaris]

Hey Bulat ,it's possible to create a cascade compression using a "p" ("preprocessor") flag in file header. Of course ,the compressor must support this .


I was thinking in a CLScript language ,but it's very complex because it can transform the data too much.

[Bulat Ziganshin]

what you mean? CLS compressors are 1st class citizens in freearc so you can build compression chains including them, like exe+test+lzma. we just lack real CLS libraries ATM

[lunaris]

what you mean? CLS compressors are 1st class citizens in freearc so you can build compression chains including them, like exe+test+lzma. we just lack real CLS libraries ATM

But is very important to differ what is a "compressed file" and what is a "preprocessed file".Using a flag in header can do this.

[Bulat Ziganshin]

i don't see practical problems with non-recognizing this in FA. but overall, we want to provide as much as possible info about CLS algo, so this may be one flags returned by "identify itself" call

[lunaris]

Is it possible to implement a plugin using zipped packages?

Mozilla firefox uses xpi packages .Xpi are zipped packages and act like plugins.

[lunaris]

Bulat?????




Did you understand?



Thanks.

[Bulat Ziganshin]

i just think that it's orthogonal question. i think about API - packaging is another question

[lunaris]

Hello Bulat

Is it possible to implement some scripting language to CLS? (Lua or python for example)

Thanks

[Bulat Ziganshin]

lunaris, what you mean? cls is just API, it may be used from any language including Lua once you wrote binding for it

i think that now any bindings will be useless since API by itself is underdeveloped and the only library implementing this API does trivial copying

[lunaris]

For example ,use lua scripts to do automated tasks in cls .For example ,use lua script to compress a lot of files.

(Like bash script for automated tasks)

[Bulat Ziganshin]

such sort of things called "Lua binding to CLS library/API"