BIT Archiver

[osmanturan]

Nothing really new but v4m was a little bit faster then f4m and f6m really slow

Yep. You are right. f6m is the slowest variant.

Ah I forgot. I used the default option (3) My commandline was only "a enwik8"

Why was my time so much under the time from m^2?

Q6600 is a monstrous CPU for BIT due to large cache Seems my contexts for mixer and/or SSE are very big. I'll try to reduce them for increasing speed. Or maybe, the reason is match model which isn't optimized much.

[Black_Fox]

Athlon 64 X2 3800+, 2GB RAM, XP SP2: EDIT: -p5

bit07   221.618 s (440 KB/s)
bit_f4m 216.825 s (450 KB/s)
bit_v4m 222.151 s (439 KB/s)
bit_f6m 241.524 s (404 KB/s)

[chornobyl]

Athlon XP 2200+ barton L2 512k, ram 512MB 266mhz, XP SP3, -p=4 330MB

07 331.344 s 294 KB/s 20894302
f4 325.609 s 300 KB/s 21198238
v4 334.797 s 291 KB/s 21204015
f6 358.157 s 272 KB/s 20891192

[osmanturan]

@Black_Fox,
@chornobyl: Thanks a lot!

Seems on all platform, f4m is the winner. I'll use it. Thanks everyone. I wonder how these variants perform on LovePimple's ancient computer (Pentium III). It would be useful to see performance on a very old platform.

[Intrinsic]

Maybe a bit late but only just saw it

E8500 @ 3265 Mhz 6Mb L2, 4Gb, XP SP3

-p=1
07  87.875s  1111 KB/s
f4  80.078s  1219 KB/s
f6  87.656s  1114 KB/s
v4  79.797s  1224 KB/s

-p=3
07  88.625s  1102 KB/s
f4  81.188s  1203 KB/s
f6  88.828s  1099 KB/s
v4  80.781s  1209 KB/s

-p=5
07  98.672s  989  KB/s
f4  86.063s  1135 KB/s
f6  93.734s  1042 KB/s
v4  85.625s  1140 KB/s

[osmanturan]

@Intrinsic: Thanks a lot!

Today I'm very happy to catch a good trade-off. Now, there is up to 32% speed gain on ENWIK8

BIT 0.7
20,823,204 bytes @ ~649 kb/s (150.416 seconds)

BIT (Under development)
21,383,162 bytes @ ~859 kb/s (113.709 seconds)

I believe that I can close the gap (even it's possible to go further) with optimizing context masks. But, I have to implement a data detection mechanism first

[Black_Fox]

I hope older platform results will still be of some use... now with slightly exotic flavour
Pentium III E 1 GHz, 512 MB RAM, ReactOS 0.3.7

bit07   631.818 s
bit_f4m 545.110 s
bit_v4m 547.285 s
bit_f6m 553.103 s

[osmanturan]

@Black_Fox: Thanks!
Your test is important for me. Even on an ancient computer, f4m is the speed winner (and also increased the gap between 0.7). So, my choice (f4m) is good even on old computers. Thanks again!

[Stephan Busch]

Hi Osman,

I have just finished testing BIT 0.7 with max settings
and can tell it nearly reaches LPAQ strength.
Should I also test any of the BIT 0.8 executables?

[osmanturan]

Hi Osman,

I have just finished testing BIT 0.7 with max settings
and can tell it nearly reaches LPAQ strength.
Should I also test any of the BIT 0.8 executables?

Wow...That's great! Thanks a lot! I'm curious about BIT 0.8 prerelease timing on your test set but I don't expect too much difference above listed benchmarks. So, if we think again that they're just some very experimental prerelease version (not final 0.8 version), there would be no reason to test on your huge test set. I only tested some trade-offs for routing a new way for future improvements and I decided to use f4m variant with extra speed-up (new mixer and no final-SSE - so, it's possible to catch >32% extra speed ) . I hope, I will finally catch >1MB/secs processing speed. Thanks again.

BTW, for a long time you were inactive. How are you?

[Stephan Busch]

So I could test f4m if you want..

Yes I was inactive for a long time..
I just needed some time on my own and was busy in other real life projects
but now I'm back and plan to release an update next weekend.
By the way: The hall of fame still lacks a picture from you. Would you mind sending me one so that I can add you to the gallery?

[osmanturan]

So I could test f4m if you want..

I'll glad to see f4m variant in your benchmark. Could you write "0.8 prerelease" as version information?

Yes I was inactive for a long time..
I just needed some time on my own and was busy in other real life projects...

If you are OK and happy, there is no problem then

but now I'm back and plan to release an update next weekend.

This is really what I was waiting for. Thanks a lot.

By the way: The hall of fame still lacks a picture from you. Would you mind sending me one so that I can add you to the gallery?

I'll send as soon as possible. Thanks a lot.

[Stephan Busch]

Hi Osman,

the f4m variant is not faster on my system and compresses 80 MB worse compared to BIT 0.7. both programs were run in -p=5 mode.

[osmanturan]

I've already expected a compression dropping due to unoptimized context masks (only high orders contexts were used: order 2-4, 6 which are suitable for text compression - not binary). But, I didn't expect worser speed Possible reason could be increased random access due to binary files with high order modeling. I'm sure next version will be much faster. BTW, I plan to put extra codecs to BIT for practical purposes: ROLZ and BWT. I hope, everyone will be happy with future improvements.

P.S: Forgive me, but I'm too curious about test results. When do you plan to update exactly?

[Stephan Busch]

I can send them by email, because the whole chart is not ready for upload now.. so you may give me your email address?

[osmanturan]

Thanks a lot. You can send it to osman att osmanturan com (I hope, the spams can't get my mail address )

[Stephan Busch]

Hi osman

Did you get my mail? A new mail I wrote today bounced back -
there must be a blocker active..

Greetings,
Stephan

[osmanturan]

I got your only one mail and just replied with a bit long content. Sorry for writing late. I was busy with two overlapped projects which must be finished until this weekend

Edit: You can try also this mail address osmanturancom att gmail com