100% Lossless Compression Theory - Please Join Me

[SmallestFilePoss]

Hello,

Introductions:
I Christopher Eklund, and I am 42 years old, and I'm American currently teaching English in China. I recently started a thread over at BitCoinTalk.org located here: https://bitcointalk.org/index.php?topic=288152.0 regarding a way I have discovered with 3 years of serious thought on the subject of compression to compress a file inside of what I would call "Virtual Containers" which are found in Nature.

The encoding process will then spit out a unique "hash" (although it's not a true hash, I'm not using standard any of the standard encryption techniques that you are familiar with) for every file you put through it. This "hash" or what I call "Crypto Key" is the file's unique digital thumbprint, and that key can be used to restore the file from out of Nature, where every possible combination of anything created already exists. What I am saying is this, you can send a Blueray file (say Lord of the Rings 3) at 50GB into this Virtual Container to obtain it's thumbprint. And then once you have the thumbprint code (the unique identifier of that file), anyone in the world can reproduce it by inputting the code into the software program I want to develop and recreate it at will. You would call this 100% lossless data compression sent to a Virtual Cloud server that exists in Nature already. You would think of it as compression, but nothing is actually compressed. The file is analyzed according to a few basic rules (it took me years to discover) to discover its thumbprint, and that is the same as copying it to a parallel universe that exists for everyone introspective of their location. A person on another planet 100 lightyears away could get a small Crypto Code and be watching Lord of the Rings 3 the same day, but no one copied the file TO him. The file was not actually compressed as you think of it as software engineers and coders. It is analyzed, and that pattern of bits it creates assign a Crypto Code which can help recreate that file anywhere in the world without having to carry the actual file on a computer or USB stick, or anything. The code can be memorized in your own head where no one else will even know you are carrying files that way. Imagine how this would help the corporate world from serious theft of their classified documents!!

I have been attacked for suggesting this was even possible, you can read the forum's contents there and see for yourself. But I am not a con artist, I truly believe this is possible and that I know how it is possible, and I would like someone to help me build the working software program so we can show that I have been right all along. In fact, I am here because some of the members did believe me enough to suggest I come here and try to talk with you, who are the most serious about this field of computer science research. Here is the original post, modified for this forum based on what I've learned along the way regarding how *I* should even see this theory. I used to think this was data compression, but what I am starting to see is that it is a file copying technique that places a file inside of a Virtual Container in Nature and by obtaining its thumbprint, one can restore the file from anywhere in the world without even owning the original file. Quite a scary idea, if it can be gotten working according to my suggestions!:

-------------------------------------------------------------------------------------------------------------------------------------------------------

The Achievement:
I have made an amazing discovery and spent a year trying to talk everyone I know into helping me with my solution for discovering the fingerprint of any file as it already exists in Nature (in the infinite permeatations of which Nature itself is capable), leaving only a basic Crypto Key that holds the seed which allows a smart software program I'm proposing the ability to re-create the entire file from scratch. Its a truly elegant solution, a truly stunning discovery worth millions of dollars possibly.

Back Story:
I'm just a teacher and don't have a fancy title or famous or powerful friends. I am just your average guy. But I like to solve puzzles, and I believed in myself, and I think that counts for more a lot. Imagine what we could do if you helped me to program this software!: With a software version of my concept, you could take your 50-gig Blueray version of Lord of the Rings 3, compress it down to a 4 Kilobyte file (the size of one nearly-blank text document) and email it to your friend in moments. What's more, imagine that this file, because it no longer contains any data at all, just a string of Crypto Keys (the chunks of data), your file is now 100% unrecognizable, scannable, or readable from any third party inbetween who might intercept your file. It's basically 100% unreadable totally compressed data. This is what I know I have achieved.

Starting in 2009, I began to have day dreams about creating something merely by thinking about it. I was looking around for ideas about something that could be created merely by thought, not a product of any kind, because I don't have those skill sets. After a while, because of my love of hackers, at first I thought I would create a game of some kind. I created an awesome game idea, one which would take all of those useless brain games and put them into an architecture that would make them important for solving a larger over-arching goal of hacking a mainframe. In the course of creating the game (and yes, this game idea would itself be worth millions if you knew everything it entailed), I came across a far more important idea, the idea of being able to compress real-life data in all computer-based files down to absolutely (just short of) nothing.

In 2011, I worked on it the whole year in my head while at my real job, my teaching job. I figured out a way to do it even then, but it was so tedious in trying to explain it to my one hacking buddy who was smart enough to be able to create the program for me that he just gave up listening to me. He wanted to help me, but I had no programming experience and wasn't able to explain the idea. I wasn't about to give up though, so I went back and thought about how to resolve my theory down to a distilled set of rules. I went from like 20 rules ... down to four. This took another year. I actually (whether you believe me or not) had a dream on December 21st, 2012 (the day the world was supposed to end) that showed me how to fix my problem with the theory I currently was working under. It was so easy, it blew my mind. Its so elegant, I believe my solution for copying all data out of a file is something that deserves to win the Nobel Prize. It might be remembered in similar fashion to Einstein or Tesla.

What Do I Want by Telling All of You This?:
I am hoping to find smart individuals who believe me about this, who have the means to start a company with me, a joint venture, of which I myself have no starting capital of any kind. I have always been poor, I come from a poor family, and have no powerful friends or allies. I'm currently trying to get started in Bitcoin mining to make a little money to finally begin forming a savings. My job is very very poor, I work in a foreign country. I'm sort of trapped here by the fact that 1) its the best job I can get with no college degree, and 2) America's unemployement is up to at least 20%, far above what is being reported by the corporate-owned media. It's too risky to return to America and try to find work. I must try to make something of this idea, because it promises to be my one and only shot at escaping my situation in life.

I am willing to share equally all incomes derived from the creation of my idea if you assume the risk. I'm hoping to find an investor who wants to take a gamble, but I am NOT asking for any money. If an investor wants to put together a team after we have had a chance to discuss the concept in full disclosure (after signing NDA's protecting the work), then you will already decided to fund the operation at that time, only AFTER hearing if it will even work.

I'm certain my idea about how to do compression will work, I have hand tested the idea numerous times, and am able to reproduce the original data merely from the Crypto-Code my theory generates. Every file has its own unique code even if there is only one bit (1/7th of a byte) of difference. That one Crypto Code can recreate the entire file, every last bit, out of thin air. It's not magic, its just a clever solution. The Bible says all things are possible for them that believe. Okay, I'm paraphrasing. But yes, I believed I could do it, and now I have it, and I'm desperate to create this and see it work!

Where Could This Idea Lead?
1) Reducing Internet Traffic Worldwide. Speeding up transfers of all datum. Imagine Sony PS4 letting you download an entire game online in mere moments, and then decompressing it to the hard-drive (might take a while to do that part) .... But the internet wouldn't suffer, it would get cleaned up in a massive way.
2) Government Uses
3) Every file sent over the internet is encrypted and totally unreadable as it is enroute (although it could be decompressed and analyzed later) but I have the ability to offer different virtual containers for every file (meaning just by choosing the container from amongst millions of possible ones) your file is encrypted too. The file can only be reconstituted properly if the Crypto Code and the base Virtual Container is known. The Base Virtual Container is the encryption protocol for your file.
4) Every file in existence can be catalogued by its resulting Crypto-Code, making storing a backup of all data online fit on one hard drive. The entire contents of the internet and every computer in the world could fit on one small hard drive in encoded form.
5) New ways to do old things, for example, recording video could be done on the fly straight to Crypto Code, leaving only a 4 kilobyte file for each video on your smart phone. When you get home, you copy the file to your main computer and it decompresses to the original-sized file.
6) Create new ways to store data that require almost no Hard Drive space for companies like Facebook and Youtube (the back ups, not the live files, because the decompression process is way too slow for real-time delivery). How many 4k files will fit on a 1TB hard drive, anyway? Each one of those files could be anywhere from 40 megabytes to 1 TB, by the way, filesize is irrelevant.

.... and just for fun:
7) One day, teleportation projects could benefit from being able to send a small packet of data that contains the entire DNA re-sequencing of the person being teleported. Star Trek could become a reality, because you wouldn't have to send billions of Terrabytes of Data through space, just send the small Crypto-Code packet (maybe 500K), requiring way less energy to transmit the actual data. Almost none, in fact.

FINALE:
There was a TV show episode about this ("Person of Interest") starring that guy from Lost ("Ben" the island leader) where they talked about someone who had done what I am claiming I have done. It was fictional. I was laughing when the TV show aired because I had already basically achieved the idea myself, but it had yet to become efficient enough to not have loopholes that would break the theory and keep it from working. Those have been resolved.

I AM ASKING SOMEONE HERE TO BELIEVE IN ME AND CONTACT ME ABOUT GOING FORWARD WITH A WORKING SOFTWARE FOR THIS!

This is real, and I need your help to complete a working prototype of the software and show the world what's been achieved here.

I welcome all of your comments.

Sincerely,

Christopher Eklund
Author of Concept

[biject.bwts]

I am surprised your posting such stuff on this site. It violates the obvious simple but true counting theorem.

However you could put every DVD movie every made and in the cloud and give each an index value that you
could pass to your friends to limit the amount of communication interaction between you and them.
But at there own pace they would have to go to cloud to get the movie.

[Sebastian W]

So you have a really big dictionary, like the number PI, a parallel universe, a perfect pseudo-random-number-generator or a yet unknown source, and create a pointer to the right position in your source? Ok, that's possible. No one can doubt that. But how long must the pointer be? (or the initialization values for a pseudo-random-number-generator?)

Let's assume, you compress 50 GB to 4 kB. Then I change one bit in the 50 GB and you compress it again to 4 kB. Your second 4-kB-code must be different to the first one to represent different 50 GB, right? Whenever the 50 GB change, your 4 kB must also change. Collisions must not happen.

But I can change my 50 GB more often than you can change your 4 kB. So collisions will happen. So it is not possible to "store" 50 GB in 4 kB.

But it is possible to copy my 50 GB somewhere and when someone asks for it and has the right key or adress, he can have access to a copy. Like Dropbox. Or even different like: "Klick this amazon link and press buy http://www.amazon.com/The-Lord-Rings.../dp/B0026L7H20"

[Matt Mahoney]

I'm not sure if you are posting to the right group. This group is for people writing data compression programs that actually work. But there is another group of like-minded people pursuing similar goals that might be able to help you with your ideas: https://www.facebook.com/groups/magic.compression/

But the group has been kind of quiet lately. I don't know if it is because most of them don't know how to write code or because they don't have enough math background to understand the counting argument. (If you don't know what the counting argument is, see http://mattmahoney.net/dc/dce.html#Section_11 ).

Also, I don't know if your algorithm depends on it, but there are 8 bits in a byte, not 7.

[SmallestFilePoss]

So you have a really big dictionary, like the number PI, a parallel universe, a perfect pseudo-random-number-generator or a yet unknown source, and create a pointer to the right position in your source? Ok, that's possible. No one can doubt that. But how long must the pointer be? (or the initialization values for a pseudo-random-number-generator?)

Let's assume, you compress 50 GB to 4 kB. Then I change one bit in the 50 GB and you compress it again to 4 kB. Your second 4-kB-code must be different to the first one to represent different 50 GB, right? Whenever the 50 GB change, your 4 kB must also change. Collisions must not happen.

But I can change my 50 GB more often than you can change your 4 kB. So collisions will happen. So it is not possible to "store" 50 GB in 4 kB.

But it is possible to copy my 50 GB somewhere and when someone asks for it and has the right key or adress, he can have access to a copy. Like Dropbox. Or even different like: "Klick this amazon link and press buy http://www.amazon.com/The-Lord-Rings.../dp/B0026L7H20"

The theory requires an irrational number like Pi to operate. Let's use Pi for an example and keep in mind, I don't propose to use this for files smaller than 500 kilobytes in size because the filesize for a completed file is 4 kilobytes.

Let's treat Pi (from the decimal point) like a timeline with every index point a possible stopping point for a sequence of data. Each bit (0s and 1s) determines where the car driving on this road will turn. Right turns change vector, but left turns keep the same vector. Thus, you can see how, given a large enough filesize, one journey along the path cannot possibly end up arriving at the same spot (the Index in Pi). To further create complexity so as to avoid running into the counting argument, only so many megabytes are encoded per what I call "Mega Chunk" let's say 500 Megabytes per Mega Chunk. Every bit causes the car to go a new direction and vector through Pi, arriving at a unique location. But even if that's not the case, the individual Chunks each have their own Crypto Keys which, when combined together in the final file output, can add to the complexity of the final file, ensuring a counting error is not found. Every time you change your 50 GB file, the reason I can still get a 4k file is that the Index location changes that your timeline would end at. Also, if there is a way to solve for counting error collisions while encoding the file, we could also return an error code "This file collides with another file, would like you like to add 1 bit to be removed later?" You say yes, and you get a unique Ending Point that does not conflict.

Furthermore, unlike other theories where a counting argument error might be the end of the output file, my theory contains a way to give the user a chance to choose between all of the possible overlapping files (should the programmers decide to go ahead with conflicts anyway). Let's say you have a 1 megabyte file. You encode it and go to your friend's house, keeping only the code in your memory. The program (this does not yet exist, which is why I need help to code it) offers you the option to put in your Crypto Key. You do so. It then says, after a few minutes, that there are six overlapping possibilities found, which one would you like to send out ie, "copy back to the desktop"? You pick the 1st one, it goes to work and returns junk, or some other file possibly. It asks, was that your file? You say no, pick another choice ... it keeps going. Finally, you find your file and it is recovered.

----
Matt, you don't sound very friendly, I am sorry for that. Perhaps you are not in the right group, the group for cynical people who have lost their humility toward others who are trying to reach out for help. I am trying to create something, I don't need your mockery. Also, I'm, sorry I made a typo, there is no way you think I meant 7 bits is there, really? Come on. I was a bit too eager to finish that long introduction and didn't catch my mistake. Thanks anyway.

[m^2]

There is pifs filesystem:
https://github.com/philipl/pifs
But it doesn't compress well...

Anyway, 2 the OP.
Matt was sarcastic, but he pointed you in the right direction: to the description of the counting argument. I suggest that you click the link.

[RichSelian]

it's hard to believe that you hold such a "great" idea for years without any practicing or proving.
most of your students have learned the permutation theory in senior middle school. Why not talking with them and find what's wrong with your 100% compression idea?

[Matt Mahoney]

Matt, you don't sound very friendly, I am sorry for that. Perhaps you are not in the right group, the group for cynical people who have lost their humility toward others who are trying to reach out for help.

Well, I'm sorry if I did not come across as friendly. I realize that not everyone has the math background to understand the counting argument. Usually when someone says that the counting argument doesn't apply to their algorithm or that there is a way around it, it usually indicates a lack of understanding. Maybe I can help. Suppose you claim that for some big enough value of n that you can compress every string of n bits to something less than n. For example, suppose n = 3. Then you have 8 possible strings you might want to compress:

000
001
010
011
100
101
110
111

To compress every string of 3 bits you have to assign a code of 2 bits or less. Each of these codes has to be different, because otherwise if two strings had the same code then the decompression program would not know how to decode it. There are 7 possible strings of 2 bits or less:

"" (empty string)
0
1
00
01
10
11

This means you can't compress every string of 3 bits because you have 1 less code than you need. If you try with n = 4, then again you have the same problem. There are 2^4 = 16 possible 4 bit strings but only 15 possible strings of 3 bits or less. (2^4 means 2 to the power of 4, or 2 x 2 x 2 x 2). If you try n = 5, then there are 2^5 = 32 possible 5 bit strings but only 2^5 - 1 = 31 possible codes. If you're not sure about this, then write them out.

Maybe you see a trend here. In fact you can prove by induction that you are 1 code short for all n, no matter how large. The number of codes of length less than n is 2^0 + 2^1 + 2^2 + ... 2^(n-1) = 2^n - 1. The left side is the number of codes of length 0, 1, 2,..., n-1, and the right side is 1 less than you need. The proof by induction is to show the statement is true for n = 1, and then prove that if it is true for n then it is also true for n + 1. So the first part is:

2^0 = 2^1 - 1

which is true because 1 = 2 - 1. To prove the second part, add 2^n to both sides and you get:

2^0 + 2^1 + 2^2 + ... + 2^(n-1) + 2^n = 2^n - 1 + 2^n
= 2(2^n) - 1
= 2^(n+1) - 1

which shows that if the statement is true for n then the statement is also true for n+1, which completes the proof.

And sorry if you don't understand the math. It's a common problem. It's why people persist in looking for a magic compression algorithm. They don't understand the proof that it doesn't exist, so they don't believe it.

[nburns]

It's a lot easier to convince people that you're right if you implement it and make it testable. It isn't really that hard to teach yourself a programming language. If you implement it and demonstrate that it works, I guarantee Matt will come around.

[Sportman]

Best Christopher, I check for almost 20 years unbelievable claims and their inventors including data compression. I can see in your story and write style that you are probably wrong where RichSelian has the best comment "it's hard to believe that you hold such a "great" idea for years without any practicing or proving.".

There was one inventor Jan Sloot https://en.wikipedia.org/wiki/Jan_Sloot who did something described by you, every movie to 1KB (wrong in Wikipedia) unique key and used a 5 x 74MB reference memory to decode it back. He gave demo's many times, but two days before deliver his main algorithm by notary office he died in his garden with heart attack. Accept some working hardware no papers where found back. He said multiple times he did not use binary storage format to store info but still had 1KB end key at 128KB key card. The 1bln dollar planned IPO was canceled and company The Fifth Force http://www.davoc.com/ is still in sleep mode in hope something useful is found back.

[SvenBent]

if the "hash" is smaller than the actual file you have less combination of hashes than files.
so you cant have a unique hash for every file.

if the hashes are not unique for each file how do you find the right file from a hash?

[msmaniac]

My 2 bits on the matter at hand:

Another way of expressing the counting argument that might be more intuitive: http://en.wikipedia.org/wiki/Pigeonhole_principle

Also this can be expressed as attempting to compress random strings of data in a dependable way for all possible permutations.
Here are some posts relating to serious and very interesting attempts to exploit characteristics of random data for compression: http://blog.adityon.com/category/data-compression/

[GOZARCK]

.................................................. ...........................

[Sportman]

(project7)

This project is a hobby project of somebody trying to make something similar as Jan Sloot SDCS (Sloot Digital Coding System).
So not Jan Sloot algorithm and also not something working as Jan Sloot system did.

English info about this hobby project:
http://jansloot.telcomsoft.nl/Source...Paper_V1.0.pdf

Jan Sloot did not told any details of his stand alone system, size of a big cigar box (without harddisk), where you could put a (128KB) smart card in with 1KB keys of pre encoded movies and play crystal sharp till 16 videos (also just recorded video of the demo scene) at the same time at any big monitor (via video out box) and jump to any place in the movies instant. Also in his patents the algorithm is not described. Only that he used 5 algorithms with 74MB memory each and CPU(s) of 300-450MHz Pentium (we talk about 1998/1999). The box also worked in US at demo's by investors while Jan Sloot stayed in Holland.

At the Davoc website is maybe the biggest hint:
"Davoc is the first non binary system. A revolution in it, bases on three colors plus a clearness factor."

[rarkyan]

my method still the best. muahahahaha

[nburns]

I wonder if there are other fields that inspire such fantastic, under-specified solutions. Network cards with infinite bandwidth that work through the ether? CPUs that are infinitely fast with 5 wires? What is it about compression that attracts ideas that don't depend on any property of actual data?

[Nania Francesco]

At the moment I'm working on a prototype of a sliding window that could give 10% improvements in the LZ77 compression or ROLZ but my results are mostly theoretical and more achievable in text files. Think to compress so much is really superior to my fantasy!