Just have an idea to add a simple delta/table transformer with auto-detection to the LZPM.
There are many possibilities. For example, here is the approach used with SQX v1.0:
1. For each small block (256-4K bytes) we count number of bytes seen with and without delta transformation. Originally SQX keeps track on:
0 - raw data
1 thru 4 - delta transform (8 - 32-bit analog data accordingly)
I have an idea to restrict to say 0 and 4 (untouched data, and delta for 32-bit tables/data)
2. We choose the transform with smallest numbers of symbols
for example:
raw data contains 200 different symbols
and
delta processed data contains just 50 symbols
we will choose to enable processing. Thus, for next block we switch our current mode to say delta4.
3. And so on, we process each small block to decide which mode use with the next one.
4. Decoder does the reverse thing - we process each block (decode if needed) and decide use or not preprocessing for the next block.
The idea is simple and relatively fast. The question is does anybody know another possibilities/methods for FAST delta encoding with auto detection?
![]()