Need some ideas to implement in a compressor

[Chirantan]

I've been a long time lurker in this forum,learned a lot from this forum.

I guess its time i work on a compressor of some sort. The compressor would mostly be used on binary data so I guess LZ would be the right choice.

There are 2 ideas i have right now:

1. LZMA with TPAQ entropy coder if its possible or any ways to improve LZMA ratio.

2. Add optimal parsing to BALZ and maybe add the entropy coder here too.

I'm open to any ideas or implementations

[Lucas]

If you're going to use a basic CM I'd recommend pairing it ROLZ as it's more context oriented than lz77 (lzma) and can achieve similar or greater ratios. That way the context mixing only has to be done on the low orders (order-0 to order-2 at most) which should help with speed. Furthermore you can sneak in an lzma-style state-map (http://mattmahoney.net/dc/dce.html#Section_523) for handling different transition states in the ROLZ model and have it influence the context mixer.

If you're set on lzma modifications the only thing I can think of to improve that would be to use a kind of dictionary preprocessor to cluster matches closer and reduce the offset size, and possibly PAQ mixing.

Good luck on your project!