Behemoth Rank Coder - BWT

[Lucas]

I am pleased to announce BRC, a fast and strong bwt postcoder I've been working on.

Source code: https://github.com/loxxous/Behemoth-Rank-Coding

Exe's: https://github.com/loxxous/Behemoth-...oding/releases

(Repost since the last thread vanished)

[SolidComp]

Yeah, what happened to all the threads today?

[Lucas]

I think the server crashed or lost power in the middle of the night and had to rollback to the last backup, unfortunately losing all 3 pages of this thread and a ton of other threads.

[JamesB]

Reattaching my rans_dynamic experiments which includes a RLE0 version. Warning, this is a total rag-tag bunch of experiments with some not even having working decoders. It's simply an exploration task, but some good may come of it.

$ cc -O3 -g -DARITH order0.16b_rle0c.c
$ ./a.out < ~/scratch/data/enwik9bwt1024.brc2 > /tmp/_
1000983528 to 169252683
Took 7098 msec, 141 MB/s
$ ./a.out -d < /tmp/_ > /tmp/_2
Took 14362 msec, 69 MB/s

BRC4 is marginally larger, as it has RLE0 already built in so this model isn't ideal, although it's faster as it has less data to encode.

Edit: actually that statement isn't correct. It's largely the same time on both brc2 and brc4 as the entropy encoder is quicker with low complexity data it seems.