PPM Compressor Small datas

[nm_m]

Hello,

I'm new to the forum and I've been researching however not found.
I wonder what the fastest ppm compressors for small data. 50 bytes for example.

Thanks

[biject.bwts]

I am not sure why you are interested in speed. Most compressors will end up expanding a small file of 50 bytes or less. So to have a chance you would most likely need a "bijective ppm compressor" One of the first and still one of the best for small files is BICOM by Matt Timmermans.

At http://www3.sympatico.ca/mt0000/bicom/bicom.html

[Piotr Tarsa]

If the input strings have enough similarities then pre-trained static PPM model could be used to compress them.

[biject.bwts]

The reality is that for random data in files of 50 bytes or less. No general compressor exists to compress the data. Sure some of the files could be compressed smaller but most files would not be compressed to a smaller size. You are better off doing nothing. Unless you know the files of interest form some sort of statistical subset of all files and you use a compressor that takes advantages of the expected subset.

[Piotr Tarsa]

There's no compressor that can compress random data :] Maybe you've meant something else, like random, but skewed probability distributions.

I think general compression algorithms won't work well on 50 bytes long independent chunks. It would be much more sensible to extract inter-block similarities either using pre-computed models from some reference input blocks or keeping a model between subsequent coding invocations.

[nm_m]

I was wrong in saying that like speed. In the case no matter the speed, I need some compressor that compresses packets of 30 ~ 100 bytes in the smallest possible packets.

[Piotr Tarsa]

What's your usage scenario? If you're trying to reduce two-way communication overhead then I suggest keeping a compressor state on both ends. You can even use zlib with flushing after each packet and you'll probably achieve some noticeable reduction in size.

Or at least provide some sample packets to analyze.

[biject.bwts]

I was wrong in saying that like speed. In the case no matter the speed, I need some compressor that compresses packets of 30 ~ 100 bytes in the smallest possible packets.

Your not giving enough information. Sometimes a fixed packet size is best. Don't forget if you are actually able to compress the packet smaller then somehow you have to be able to send the new packet length.

To be able to compress at all you need to take advantage of the structure of the data. You also have to realize that many times the size of the packet will increase. You may have 30 to 100 bytes for input but then you may have to live with 20 to 110 bytes at output.

It might even be better if several packets are combined to one big packet for compress and then broken apart after decompression. Tell us more.

Look one could easily modify a 256 symbol bijective arithmetic coder to take a 100 byte file and if all bytes are different it would expand to its max of 110 bytes ( or what max your heart desires ) and then files with a lot of repeated characters would compress smaller than 100 bytes. The larger you allow for the maximum file expansion then the smaller a file with many repeats would compress.

[nm_m]

Will be used to compress data TCP protocol. And packet size, I'm talking are between 10-100 bytes. I would like to reduce this size with a compressor.

[nburns]

Will be used to compress data TCP protocol. And packet size, I'm talking are between 10-100 bytes. I would like to reduce this size with a compressor.

A lot of the packets, especially ones that are minimum in length, are probably mostly canned data that's dictated by the protocol, like ACK messages. The best way to approach those is to replace them with shorter codes, such as with Huffman. The reason shorter messages aren't already built into the protocol, though, is likely because compression would slow down the performance of the network.

[Piotr Tarsa]

Compressing packets independently won't allow you to exploit similarities like repeated IP numbers, port numbers, etc between packets.

You still didn't tell us whether you need to compress the packets indepenently or in blocks.

Also efficient compressing of structured data requires some special treatment from the compressor. Parsing TCP packets shouldn't be hard but I haven't heard of any compressor that does that.

[nburns]

Compressing packets independently won't allow you to exploit similarities like repeated IP numbers, port numbers, etc between packets.

You still didn't tell us whether you need to compress the packets indepenently or in blocks.

Also efficient compressing of structured data requires some special treatment from the compressor. Parsing TCP packets shouldn't be hard but I haven't heard of any compressor that does that.

True, even ACK packets would have parts that vary, such as IP addresses. So they couldn't be replaced 1:1 with static codes. But I think once you've separately dealt with things like addresses, you'd be left with a message saying "ACK." That's pretty much exactly the same concept as the kind of atomic symbol with associated probability that Huffman was invented to make codes for.

Let's not get too deep into possible implementations, though, because the parts of this scheme that we know about don't add up to anything workable, as far as I can tell. You couldn't just compress TCP packets and send them back down the wire in compressed form and expect a functioning network. So, I think, either we have much too little information about the project to possibly start coming up with solutions, or perhaps the question was misinformed.