My new compression algorithm

Hello, i recently invented a new algorithm - i think it is new because it is not basic on entropy, or data repeating, like LZ family or arithmetic encoding. Because i'm not a software developer so i currently haven't created compression application yet. All information below is only my prediction
My algorithm have some advance:
1. Can compress ALL TYPE OF FILE " include files compressed by itself ", so that we can perform multi level compression to maximize compress ratio (compress ratio can be 100:1 if necessary)
2. compression speed is not affected by the input string, faster than arithmetic but slower than lzw... moreover, my algorithm do not consume too much memory and cpu as PQA ( it just nearby LZ family algorithm), the more memory and cpu speed, more compression speed and ratio we gain.
3. decompression speed is faster about 4 times than compress speed. (we compress once to decompress millions times)
4. security feature: encoding sequence will create an string of controller that length 1/1000 of source string and data cannot be decompressed if we hide that string
.Disadvantage:
Low speed: every compress - decompress level or circle take time equal to LZ family but only can reduce less than 5,5%(can be increase in future research ) of source string, because we are compressing the incompressible data

Is it valuable enough to develop a application?

Quote:

---

Originally Posted by tefara

Is it valuable enough to develop a application?

---

No.

Sorry, couldn't resist the temptation to reply sarcastically.
Long answer:
Math doesn't lie. Your algorithm may be able to compress incompressible data. But it won't be able to decompress it back. Your program can't work which implies that it won't work. If you don't believe that it can't work, (and you probably don't), I suggest that you read the literature:
http://mattmahoney.net/dc/dce.html#Section_11
If you don't believe the literature either, think about creating a decompressor.
If you still think you have designed a correct decompressor, implement it.
If you have implemented a decompressor and believe it's correct, develop a compressor too.
You should find soon after that the couple doesn't always work.

too young too simple.:D

Quote:

---

Originally Posted by tefara

decompression speed is faster about 4 times than compress speed. (we compress once to decompress millions times)
Is it valuable enough to develop a application?

---

Do you guy read my post carefully? any compression algorithm have to go with decompression algorithm or it will be useless.

Quote:

---

Originally Posted by tefara

Do you guy read my post carefully? any compression algorithm have to go with decompression algorithm or it will be useless.

---

No compressors can compress all types of data. Whan you say you invent such an algorithm, it means you lack some knowledge of math.

In the past, noone believe that the earth is round, but in-fact, it's round. Never said something impossible. My question is only the relative compression ratio and compression speed. If compare with LZW algorithm which have compression ratio about 30% for English text file, my algorithm at the same time can only reduce 5%. But, the advantage here is re-compressible to create infinite compression ratio

Quote:

---

Originally Posted by tefara

In the past, noone believe that the earth is round, but in-fact, it's round. Never said something impossible. My question is only the relative compression ratio and compression speed. If compare with LZW algorithm which have compression ratio about 30% for English text file, my algorithm at the same time can only reduce 5%. But, the advantage here is re-compressible to create infinite compression ratio

---

it's no matter that you implement your compressor and bring it here to blame us. But i think no one is interested in helping you implement it because we all know it is impossible.

i'm not blaming any one. it is just a question.I don't try conviene you guy to believe me, i only want to ask a question, if my algorithm is real, basic on that speed and ratio, is it valuable enough to develop an application? However, great to hear you said that "it is impossible", so that i can become the first one who make it become possible. i asked this question because i hear rumor that kgb archiver can compress 600 mbs winxp disk to 1,5mb.
spoil a example, i can encode and decode a string of 256 totally random char ( all of them are different each other)
input string 256 char 8bit = 2048 bits
output string 240 char 8bit = 1920 bits

tegara, if i will find proof that 0=1, what should i do next? apply for PhD or search errors in my proof?

> so that i can become the first one who make it become possible

actually people like you arrives to compression forums every month or so. the comp.compression newsgroup was filled by them some years ago, i don't know about current situation

> i can encode and decode a string of 256 totally random char

do you mean that you can encode ANY 256 bytes to 240 one? it's impossible, hope you understand it. and if you can encode only some of 256-byte sequences to 240-byte, it's very easy to compute how much of those 256-byte strings can be encoded to 240-byte. please make such computation for me

The problem is (as others have pointed out), if you can compress all strings, then you cannot decompress them all to the original string, because at least some strings have to compress to the same shorter string, and given that shorter compressed string, you cannot know which longer string it originated from.

For arguments sake, consider a function that can compress all integers from 1 to 1000 to an integer between 1 and 10. At least two of the integers from 1 to 1000 have to compress to the same integer between 1 and 10 (actually a lot more of course). Say both 600 and 700 compress to 5. Now somebody gives you a compressed 5 -- how do you decompress it? to 600 or 700?

And just to note, it is unlikely that 256 random bytes are all different from each other.

Quote:

---

Originally Posted by tefara

i'm not blaming any one. it is just a question.I don't try conviene you guy to believe me, i only want to ask a question, if my algorithm is real, basic on that speed and ratio, is it valuable enough to develop an application? However, great to hear you said that "it is impossible", so that i can become the first one who make it become possible. i asked this question because i hear rumor that kgb archiver can compress 600 mbs winxp disk to 1,5mb.
spoil a example, i can encode and decode a string of 256 totally random char ( all of them are different each other)
input string 256 char 8bit = 2048 bits
output string 240 char 8bit = 1920 bits

---

there are 256^256 different 256-byte strings, and only 240^256 different 240 one.
you compress all 256-byte strings to 240 byte, that means at least (256^256 - 240^256)=3.23e+616 output results are duplicated. how can you recover two same 240-byte compression outputs into two different 256-byte inputs?

split source string into block of 256 char string
if all of char in every block are different, the result is best 256 char to 240 char ( that can be better if using another algorithm an a supporter)
if they are not totally different, a dynamic dictionary that lenght 32 char is required to decode, if 32 char dictionary can be used for more than 3 blocks...boom we gain some compression ratio

Quote:

---

Originally Posted by RichSelian

there are 256^256 different 256-byte strings, and only 240^256 different 240 one.
you compress all 256-byte strings to 240 byte, that means at least (256^256 - 240^256)=3.23e+616 output results are duplicated. how can you recover two same 240-byte compression outputs into two different 256-byte inputs?

---

we are encoder, don't something stupid like this ok. if keep thinking like you, there is no compression algorithm on the earth can work. Moreover, practice your English and math skill some more please!

Quote:

---

Originally Posted by tefara

spoil a example, i can encode and decode a string of 256 totally random char ( all of them are different each other)
input string 256 char 8bit = 2048 bits
output string 240 char 8bit = 1920 bits

---

You can do better: if I'm not wrong, a string of 256 totally random char (all of them are different each other), can be compressed down to 1683.9962... bits (211 bytes) = log2(256) + log2(255) + log2(254) + ... + log2(1).

Quote:

---

Originally Posted by tefara

we are encoder, don't something stupid like this ok. if keep thinking like you, there is no compression algorithm on the earth can work. Moreover, practice your English and math skill some more please!

---

yes there is no working "universal" compressor on the earth. all compressors we inverted only work on redundant data.

it easy to proof that your algorithm is impossible:

Code:

---

Y=YourCompressionAlgorithm(X)
X∈Sx
Y∈Sy

---

for "compressor that can compress all kind of data", card(Sx) must be LARGER than card(Sy). in this way, you can NEVER find X=YourDecompressionAlgorithm(Y).

i am chinese and speak suck english, sorray about that.:D

Your brain has been hacked!!!, All type of data are redundant data, if we has a string of 257 char_8bit, there must be at least one char is repeated and it is the redundance we need to compress in some way. Once again, i don't care if you believe or not, i just wonder about the relative of speed equal to LZW but only can get around 5% reducing is enough or not, so dont talk something useless please

and if we have sequence of 3 bits, either 0 or 1 is repeated too, so it also can be compressed? :)

Great to hear that anyone believe it ís impossible, that's mean my algorithm is unique. But i'm dissapointed that you guy always try to hacked into other people brain instead of thinking seriously my question about compression speed and ratio. Topic close here until i has finished study phyton and develop my software. A video or some pictures about my work will be upload later

it was rather easy case, colleagues :D

Quote:

---

Originally Posted by tefara

Great to hear that anyone believe it ís impossible, that's mean my algorithm is unique. But i'm dissapointed that you guy always try to hacked into other people brain instead of thinking seriously my question about compression speed and ratio. Topic close here until i has finished study phyton and develop my software. A video or some pictures about my work will be upload later

---

all of us believe (and proof) it is impossible. once you implement it, i believe you will be the richest man in the world (at least you can get all bonus of hutter price):D

Quote:

---

Originally Posted by RichSelian

all of us believe (and proof) it is impossible. once you implement it, i believe you will be the richest man in the world (at least you can get all bonus of hutter price):D

---

WOW, thank so much, i always try to find some website like that since i invented my algorithm, so all i need to do is compress the enwik.zip to smaller than 34.8mb, too simple.:D

Quote:

---

Originally Posted by Bulat Ziganshin

it was rather easy case, colleagues :D

---

Until the next one comes around.
There seems to be a near constant flow of this kind of people with big dreams and no understanding of math.

Quote:

---

Originally Posted by tefara

WOW, thank so much, i always try to find some website like that since i invented my algorithm, so all i need to do is compress the enwik.zip to smaller than 34.8mb, too simple.:D

---

Too bad you did not listen to what people are trying to tell you. Start by reading about compression before jumping to conclusions.
Being skeptical is smart, being constantly wrong is not.

Actually the hard part is not compressing enwik8 to 15mb, it is decompressing it again.

I think if you've actually considered the comments in this thread about universally compressing random data, and still feel there is merit to your method, you should definitely go ahead and implement it. It will be a good learning experience.

http://aiz.free.fr/popcorn_mj.gif

BARF does it better, at least on Calgary corpus, but I think there is some little catch... :_shuffle2:

Another viable option is to read source file, encode some part as filename, write it to filesystem as zero-sized file. Repeat it until all data were encoded to filenames. Decode is the reverse of this process. At the end of day, you can store your 20 megabyte file into bunch of zero sized files. And their total size still going to be 0 bytes. Sounds like a bargain, right?! :cool: But once more, there is some little catch...

actually, if you have a good memory, you can compress file of any size into empty file. and it will have good old 8.3 name

Well, i used javascript to generate a string about 20 ramdom char_3bit ( number between 0~7) , using my algorithm to encode ( by hand), it and i have a string of 18,3 char_3bit. Then, i decode and get exactly the original string, I don't care you guy believe or not. I create this topic to find out that is there any other algorithm can compress random data, and discus about compression speed and compress ratio.

Quote:

---

Originally Posted by tefara

WOW, thank so much, i always try to find some website like that since i invented my algorithm, so all i need to do is compress the enwik.zip to smaller than 34.8mb, too simple.:D

---

Well, first you need to learn to write code. Then your second problem is you will discover your program doesn't work.

Really, we are trying to help. But if you want to be stubborn about it, then you're on your own.