About filter

[vcore]

Hello all,

I am curious about how does the filter work in a compressor?

For instance, some compressors have the EXE filter for improving compression ratio for executable files. And some have the BMP filter for bmp files...etc.

How the the filter work?

Can somebody give a brief explaination?

Thanks a lot!

[Nania Francesco]

Delta Encoding for BMP and Wave files!
in this forum
http://www.encode.su/forums/index.ph...um=1&topic=575

[encode]

Can somebody give a brief explaination?

Filter aka Preprocessor works as follows:

Compression:
[Uncompressed Data] -> [Filter / Pre-Processor] -> [Compressor] -> [Compressed Data]

Decompression:
[Compressed Data] -> [Decompressor] -> [Filter / Post-Processor] -> [Decompressed Data]

In other words, filter transforms the data into more compressible form. Usually the transformed data has the same or even larger size.
The compressor even "dont knows" that it compresses a preprocessed data.

For example, with EXE files, the filter translates the relative CALL/JMP instructions into absolute values.

MM filter applies delta and other tricks.

And so on...

[Shelwien]

> I am curious about how does the filter work in a compressor?
> For instance, some compressors have the EXE filter for improving compression
> ratio for executable files.

Well, sometimes, like with x86 code, its possible to transform the
original data into something better fitting the compressor's model.
Eg. jump/call targets in x86 instructions are encoded as a difference
between real target address and instruction's end address, so the
5-byte calls to the same function at different locations are encoded
by different byte sequences. But if you'd use a filter which
adds current offset to 32bits following the E8/E9 prefix
(and subtracts after decoding), the filtered data would suddenly
become more compressible, with longer matches etc.
http://compression.ru/sh/flt32.rar

Also there're similar techniques for text preprocessing, see
http://www.ii.uni.wroc.pl/~inikep/research/WRT/

Basically the idea is that preprocessor/filter turns some specific
correlations known from the file format into the form usable for
"univeral" compressor.

> And some have the BMP filter for bmp files...etc.
> How the the filter work?

Now that's kinda other thing. As bmp files are 2D tables, and
universal compression methods are 1D-oriented, so nothing much
could be done, though subtracting some estimation by neighbouring
points might be of some help.

So the correct solution is to write a custom context model for
each specific format.

Edit: well, two answers are better than one

[vcore]

Hi, thanks very much, encode, Shelwien and Francesco!