( almost ) unlimited compression with no data loss

[uglyvan]

hi everybody

well at least it works in theory;

here's the topic :

any given text file usually gets compressed to 40% of original size;

a given text file uses approximately 32 characters;

it is easy to tranform any binary file to a text file ( that is to say it uses 32 different values ) ;

for this we use a second file which will indicate what group of 32 characters we refer to

7 for a value between 224 et 255
6 pour une valeur comprise entre 192 et 223
5 pour une valeur comprise entre 160 et 191
4 pour une valeur comprise entre 128 et 159
3 pour une valeur comprise entre 96 et 127
2 pour une valeur comprise entre 64 et 95
1 pour une valeur comprise entre 32 et 63
0 pour une valeur comprise entre 0 et 31


ok.
so this file holds 8 different values
each value can be coded with 3 bits , for a total of 3/8 = 37.5 of the original file

let's take a file of 100ko
as mentioned earlier, a text file is usually reduced to 40% of original size

our output file will weight 40%+ 37.5% = 77.5%

where is the error ??

[Matt Mahoney]

Have you tried it? You're just moving 3 bits of each byte from one file to another. If you start with random data you won't get 40% of original size with 5 of 8 bits remaining. You'll get 62.5% with a good compressor.

[SvenBent]

Text is more than just having 32 different characters. Its the structure as well which you will not have when converting binary into "text" so you 40% compression rate is based on something complete different than what you are making

[Sportman]

With http://encode.su/threads/2480-ble-(f...ength-encoding you can see an example what can happen with two random files, bytes become smaller but included the (bookkeeping) bitmap and headers it become larger:

ble 7 AMillionRandomDigits.bin AMillionRandomDigits.ble7

bitmap from 75,200 to 75,199 bytes (0.00%)
bytes from 415,313 to 364,415 bytes (-12.26%)
encode from 415,313 to 451,365 bytes (8.68%)

ble 15 AMillionRandomDigits.bin AMillionRandomDigits.ble15

bitmap from 40,960 to 40,960 bytes (0.00%)
bytes from 415,313 to 389,359 bytes (-6.25%)
encode from 415,313 to 430,344 bytes (3.62%)

ble 7 2012-08-30.bin 2012-08-30.ble7

bitmap from 189,344 to 189,338 bytes (0.00%)
bytes from 1,048,576 to 920,087 bytes (-12.25%)
encode from 1,048,576 to 1,139,011 bytes (8.62%)

ble 15 2012-08-30.bin 2012-08-30.ble15

bitmap from 98,304 to 98,304 bytes (0.00%)
bytes from 1,048,576 to 983,044 bytes (-6.25%)
encode from 1,048,576 to 1,081,408 bytes (3.13%)

[uglyvan]

thank you for your contribution

that's just what I was lookin for ..