Kitty file compressor (Super small compressor)

[snowcat]

Hello,

This is my second file compressor, actually it's an upgrade from my old Bit6.
I call it super small because it only use 768 bytes RAM, fit almost any MCU (the only problem is 32 bit arithmetic in MCU :P )
I include with an ASM version, and all of the compress / decompress engine cost only 280 bytes (x86 484+ code), small even like JQcoding with better compression, so it even can be use in virus :P

It use 512 bytes for an modified StateMap from SR2, and 256 bytes to predict the next byte from the last byte of context.

Some result:
enwik8: 100,000,000 -> 59,952,178
enwik9: 1,000,000,000 -> 602,865,931

JQcoding: (http://encode.su/threads/2157-Lookin...ll=1#post43099)
SR2: (http://mattmahoney.net/dc/#sr2)

[comp1]

Quick test with my WIN95 corpus:

WIN95.SHAR       | 58,194,746 | 
WIN95.KITTY      | 33,326,515 | 5.93 s
WIN95.SR2        | 26,036,376 | 2.48 s

[Gonzalo]

Good one. But a bit slow for its low requirements, don't you think?

D:\>kitty c TeamViewer_Setup_es.exe TeamViewer_Setup_es.exe.kit
0 -> 5 in 0.00 sec.

I don't see the reason why kitty doesn't want to claw this one, neither 7z1500.exe...

Plus: a stdin-stdout release would help a lot in testing. The idea is to use it with shar, for example.

[snowcat]

Oops, I set the wrong compile setting (-Os) :P
It should be: -march=i686 -Ofast -m32 -fno-exceptions -fno-unwind-tables -fomit-frame-pointer

And seem like MinGW is better than TDM-GCC :P

Result on Corpus:

bib: 111261 -> 70454 (5.0659) in 0.01 sec.
book1: 768771 -> 445732 (4.6384) in 0.06 sec.
book2: 610856 -> 355339 (4.6537) in 0.05 sec.
geo: 102400 -> 68534 (5.3542) in 0.01 sec.
news: 377109 -> 232413 (4.9304) in 0.03 sec.
obj1: 21504 -> 13323 (4.9565) in 0.00 sec.
obj2: 246814 -> 143938 (4.6655) in 0.01 sec.
paper1: 53161 -> 31703 (4.7709) in 0.00 sec.
paper2: 82199 -> 47772 (4.6494) in 0.00 sec.
pic: 513216 -> 71673 (1.1172) in 0.01 sec.
progc: 39611 -> 23453 (4.7367) in 0.00 sec.
progl: 71646 -> 38084 (4.2525) in 0.01 sec.
progp: 49379 -> 26144 (4.2356) in 0.01 sec.
trans: 93695 -> 53756 (4.5899) in 0.01 sec.

@Gonzalo: add an error report, may you retest it?

[comp1]

Oops, I set the wrong compile setting (-Os) :P
It should be: -march=i686 -Ofast -m32 -fno-exceptions -fno-unwind-tables -fomit-frame-pointer

And seem like MinGW is better than TDM-GCC :P

Result on Corpus:

bib: 111261 -> 70454 (5.0659) in 0.01 sec.
book1: 768771 -> 445732 (4.6384) in 0.06 sec.
book2: 610856 -> 355339 (4.6537) in 0.05 sec.
geo: 102400 -> 68534 (5.3542) in 0.01 sec.
news: 377109 -> 232413 (4.9304) in 0.03 sec.
obj1: 21504 -> 13323 (4.9565) in 0.00 sec.
obj2: 246814 -> 143938 (4.6655) in 0.01 sec.
paper1: 53161 -> 31703 (4.7709) in 0.00 sec.
paper2: 82199 -> 47772 (4.6494) in 0.00 sec.
pic: 513216 -> 71673 (1.1172) in 0.01 sec.
progc: 39611 -> 23453 (4.7367) in 0.00 sec.
progl: 71646 -> 38084 (4.2525) in 0.01 sec.
progp: 49379 -> 26144 (4.2356) in 0.01 sec.
trans: 93695 -> 53756 (4.5899) in 0.01 sec.

@Gonzalo: add an error report, may you retest it?

Ahh now it's speed is correct!

WIN95.SHAR       | 58,194,746 | 
WIN95.KITTY      | 33,326,515 | 2.93 s
WIN95.BIT6       | 34,214,509 | 5.80 s

I would like to test JQCoding program as well but I don't find compiled Windows executable.

[Gonzalo]

Oops, I set the wrong compile setting (-Os) :P
It should be: -march=i686 -Ofast -m32 -fno-exceptions -fno-unwind-tables -fomit-frame-pointer

And seem like MinGW is better than TDM-GCC :P

Result on Corpus:

bib: 111261 -> 70454 (5.0659) in 0.01 sec.
book1: 768771 -> 445732 (4.6384) in 0.06 sec.
book2: 610856 -> 355339 (4.6537) in 0.05 sec.
geo: 102400 -> 68534 (5.3542) in 0.01 sec.
news: 377109 -> 232413 (4.9304) in 0.03 sec.
obj1: 21504 -> 13323 (4.9565) in 0.00 sec.
obj2: 246814 -> 143938 (4.6655) in 0.01 sec.
paper1: 53161 -> 31703 (4.7709) in 0.00 sec.
paper2: 82199 -> 47772 (4.6494) in 0.00 sec.
pic: 513216 -> 71673 (1.1172) in 0.01 sec.
progc: 39611 -> 23453 (4.7367) in 0.00 sec.
progl: 71646 -> 38084 (4.2525) in 0.01 sec.
progp: 49379 -> 26144 (4.2356) in 0.01 sec.
trans: 93695 -> 53756 (4.5899) in 0.01 sec.

@Gonzalo: add an error report, may you retest it?

Now it's better

Anyway, the error still appearing:

D:\>kitty c 7z1500.exe 7z1500.exe2
7z1500.exe: 0 -> 5 (1.#INF) in 0.00 sec.
c: Invalid argument

As today is related only to these two files... Weird

PS: Remember stdin - stdout...
It shouldn´t be difficult to add a speed counter, to make the program's output more useful. You have right now the input size and the time so it's a just a division.
Also, I feel that most users/testers might prefer % instead of bpb... But it's just a mater of taste.

[snowcat]

@Gonzalo:
As your suggestion, I made the real-time speed counter
Could you try to create a new folder, copy Kitty and 2 files to, and then try again? I think MS hate to allow to write at root of disk like D:

[Gonzalo]

@Gonzalo:
As your suggestion, I made the real-time speed counter
Could you try to create a new folder, copy Kitty and 2 files to, and then try again? I think MS hate to allow to write at root of disk like D:

Oh damn! It wasn't your problem. It seems like the OS messed up the two files. Nobody can read them. So don't bother trying to fin a bug, for it isn't there. Congrats for the UI improvements!