Hi, I'm looking for a lz style compressor or a tool which creates/uses an optimal static dictionary for matches instead of a sliding window. Currently I'm using a quite simple LZ style compressor with a 256 byte rolling history buffer, but this small window hurts compression a lot. The target system is a c64 with only 64kb ram. The files I need to shorten are prepacked using a special RLE compressor. I only decompress one byte at a time which won't be saved to memory and I can't use previously decompressed data for a sliding window. I'm using this encoding right now:
%xyyyyyyy x=0 copy up to y 127 literals from input / x=1 match length up to y 127 bytes and the following byte is the distance to the rolling history buffer
My idea is to get rid of the history buffer and use a static match dictionary which holds all compressable strings and decode using this:
%xyyyyyyy x=0 copy up to y 127 literals from input / x=1 y match len up to 127 and the next 2 bytes are the index to the dictionary which won't get larger than 64kb
The compressed data should only exist of uncompressable literals and matches to the static dictionary. Zstd has an option to create a dictionary, but it refuses work with a too small dataset. My files are between 10 and 60kb small. SmalLZ4 can use a dictionary, but I didn't find a tool to create one.
Any ideas regarding this topic?