BCM v0.10 is here!

[encode]

What does it mean? Do you plan to integrate BCM into PIM?

It's not PIM. It's a BCM with GUI. PIM should be completely rewritten to be OK with Windows 7. This one initially made for Windows 7. I really disappointed with PIM archiver - no user feedback...

[chornobyl]

I used PIM to extract quake .pak files, it works flawlessly.

[Matt Mahoney]

I added bcm -b477. http://mattmahoney.net/dc/text.html#1686

[encode]

Thanks a lot!

[encode]

BCM 0.11 will be released! Firstly, I've found how to improve compression with no speed impact at all. Secondly, binary compression for files a la "calgary.tar" can be notably improved - BCM should perform better - check out the BCM v0.08 - it's not that much complex than the latest v0.10... Anyway for a few months I will dig into FCM (BWT-output encoder) to find out what kind of stuff really works, being not computationally expensive and with low memory requirements. Basically, all latest BCM's except of v0.09 perform really well even on weak hardware such as my Atom(TM) based Netbook - the main CM model fits just in a few hundreds of KB, which is pretty small... As a note, it's a complete rewrite of BCM, and new BCM will be released as 32-bit version only. Also, it's developed mostly under my Netbook. Although all optimizations performed under 4 computers linked together via WiFi...
It's sad but a few months of work on PPMX lead to nothing. Yep, new PPMX is a few times faster, with better compression, but it's not that interesting as BCM. Although it has some advantages over PPMd, anyway, generally speaking PPMd in most cases still faster and has better compression. I tested PPMX with Interpolated SSE on R0 symbol - nice, compression on number of files became better compared to PPMd, but on texts it's not that improved. Probably SSE on MPC will help here. Anyway, my impressions that PPM is something from the past - better to write something a la M1 or continue to work on BCM - just spread my programming energy on something more interesting and promising...

[kampaster]

encode thanks a lot for BCM 0.10
I wait BCM 0.11!!!!

[encode]

Just assembled my new powerhouse PC. Since it's impossible to optimize even simplest CM using a Netbook. New PC has 4-core CPU, fast RAM, and a few fast HDDs. And I will start writing from scratch a new BCM. I've collected many ideas for the CM part - it's time to test them all!

[encode]

After a few weeks of uninterrupted optimization I've found some interesting stuff... New BCM 0.11 is the fastest of all BCMs and yet the most efficient. Each new version I tried to introduce something new. BCM 0.05 through 0.09 is one line. Version 0.09 is the most strongest and most computationally expensive - thus it's slowest. Since version 0.10 I started new conception - more light and fast CM part - to be more faster and yet provide nice compression results. BCM 0.11 is something really new - I continued that line to simplicity making BCM even more simpler and thus faster and at the same time kept proper CM performance in terms of compression power. In addition, during parameter optimization I consider bunch of test files, unlike previous versions, then I optimized a CM part on just two or three files. Yep, the optimization time is a few weeks long on a powerhouse PC, but I've found a nice parameters, trying to be more universal - apart from great text performance, keeping very good performance on binaries, highly compressible files, etc. etc. At the same time the simplicity was the new goal. I still continue programming, this time mostly on command-line user interface and other guts. Just I want to move my BCM to a definitely new level of performance and quality! To be continued...

[Skymmer]

Sounds very intriguing. It will be very nice to test it.

[Sanmayce]

Hi encode,

At the same time the simplicity was the new goal.

Yes, I agree, simplicity and definite goals are, in my opinion, the formula for a true utility.

I hope that you can create a strong console text smasher which, as you say, will be kick-ass.
I am ready to test and use your tool,
good luck.

[encode]

Since making too heavy BWT compressor is bad idea. The catch of BWT in its performance - compression power vs compression time. Instead of an overloaded BWT, better use a pure CM! Thus we should keep BWT compressors reasonable fast and efficient! Anyway, at the same time, we should not keep too many air in the compressed stream - we must provide really nice compression. So, the balance between the compression power vs compression, and most importantly, decompression time is the goal!

[Gribok]

After a few weeks of uninterrupted optimization I've found some interesting stuff... New BCM 0.11 is the fastest of all BCMs and yet the most efficient. Each new version I tried to introduce something new. BCM 0.05 through 0.09 is one line. Version 0.09 is the most strongest and most computationally expensive - thus it's slowest. Since version 0.10 I started new conception - more light and fast CM part - to be more faster and yet provide nice compression results. BCM 0.11 is something really new - I continued that line to simplicity making BCM even more simpler and thus faster and at the same time kept proper CM performance in terms of compression power. In addition, during parameter optimization I consider bunch of test files, unlike previous versions, then I optimized a CM part on just two or three files. Yep, the optimization time is a few weeks long on a powerhouse PC, but I've found a nice parameters, trying to be more universal - apart from great text performance, keeping very good performance on binaries, highly compressible files, etc. etc. At the same time the simplicity was the new goal. I still continue programming, this time mostly on command-line user interface and other guts. Just I want to move my BCM to a definitely new level of performance and quality! To be continued...

Do you have a plans to release bcm with source code?

[Sanmayce]

So, the balance between the compression power vs compression, and most importantly, decompression time is the goal!

Music, music for my ears...

[m^2]

Music, music for my ears...

Totally agree. In almost all cases asymmetry is good.

[encode]

Do you have a plans to release bcm with source code?

Nope. I'm keeping BCM as an experimental and closed source program - a la test-bed for my algorithm (BWT+FCM) and mainly for my FCM part - a highly tunable and flexible Fast CM. Since FCM is a commercial thing - I may not release it as open source anyway, at least for now...

[encode]

BCM 0.11 is mostly done! Now, I'm carefully testing the program. New version is a complete rewrite. I even changed my programming style. All in all it is the best console program I have ever made! I think within one or two weeks it will be released. Since the new program is the most stable and quality work ever made by me, it needs an extra testing. Additionally, BCM 0.11 introduces extended error checking, new command-line interface and many optimizations & improvements. The output of the program looks like this:

c:\Test>bcm -b1 mptrack.exe
BCM 0.11 Experimental BWT compressor
Copyright (C) 2010 Ilia Muraviev

Compressing 1.0 MB block...
Compressing 0.1 MB block...
100%

1159172 -> 504165 in 0.3 sec

c:\Test>