Recursive data compression patent for sale

[Matt Mahoney]

Really, I am not making this up. US patent 7733249 describes a method of compressing data more than once to improve the compression ratio. According to the patent, the method is to divide a data stream into objects and to "compress" them by treating objects as binary numbers and encode them as arithmetic expressions that have the same value. You can then apply the method recursively to both individual objects and to the stream as a whole.

You can buy the patent now for US $275,000 or wait until the auction on Nov. 11. Working code is not included.
http://icapoceantomo.com/item-for-sa...mpressing-data

Of course this is not the first time the USPTO has allowed patents on random data compression.
http://www.faqs.org/faqs/compression...section-8.html (section 9.5).

[giorgiotani]

Do they accept payments in Enron stocks or in Argentine bonds?

[m^2]

Hmm...got to sell my parcel on Mars.

[biject.bwts]

Maybe Euclid Discoveries could add this to there extensive patent portfolio greatly increasing the value of there shares for only peanuts.

[Sportman]

There are a lot with name Qin Zhang working in computer related jobs but I guess it's this women:
http://www.linkedin.com/pub/qin-zhang/9/374/358
http://twitter.com/qinzhang
http://www.booloon.com/lifetype/
http://www.facebook.com/people/Qin-Zhang/1500605029

More about her work and patents:
http://www.cs.ucla.edu/idm/2008.html#081031

Because inventor patent is located at the same address as laywer:
Zhang, Qin.
Qz Consulting
415 S Oxford Ave., # 304
Los Angeles, CA 90020

100% sure now, found with P.O. box and ZIP code listed in Due Diligence patent request papers:
QIN ZHANG
P.O. Box 66309
Los Angeles, CA 90066

In FundRace:
http://fundrace.huffingtonpost.com/

Los Angeles Qin Zhang attorney
self Q2-2009 PO BOX 66309.
Los Angeles, CA $1,000 donation to dnc services corp./dem. nat'l co

Qin Zhang entrepreneur/attorney
booloon, inc Q3-2009 PO BOX 66309.
Los Angeles, CA $500 donation to dnc services corp./dem. nat'l co

[willvarfar]

kind of unfair that inventing stuff like this is likely to be the only place in compression where the money is to be made?

you must admire the cleverness

[biject.bwts]

Will I called for info on the auction. I told the guy I always wanted a cheap patient
on some form of compression. The only patient I hold is one jointly with the US
Navy I think that means I never see a penny. Anyway the guy stated the starting
bid is 125,000 and it is "LOT 53"

Dean Becker
561.309.0011
dean.becker@us.icap.com

Dean is the guy selling it I wanted to but a bid in for 5 dollars but he would not
let me. I use to call a lot a 800 numbers when I had a landlline phone know all
I have is a cell phone not texting. But I recently got a free google phone its free
and neat all calls to US are free. I think if you play the game correcrtly maybe
you can use it anywhere.

[Matt Mahoney]

Better yet, offer your services as a data compression expert who can advise them on what the patent is actually worth.

[biject.bwts]

Better yet get Fu Siyuan to talk to her. I don't get along well with lawyers. At one time when I worked for the DOD they warned
us about pretty Chinese girls. I always hoped one would seduce me. But that was years ago and the secrets I knew are to old
so I doubt she would give me the time of day now. Don't you think they realize what its worth and that some rich dummy
is going to bye it. Don't forget I did offer 5 bucks. I would just like to know who is going to pay for it.
My only thought is that she is donating money to the dnc so maybe she already had a buyer in the government. Why else
would she give money to the dnc. I suspect that since the ones with the cash in the government aren't rocket scientists so
they most likely belive what ever there told by a sweet voice.
Matt you have the Phd not me and I suck at writting a resume. So why don't you pick up a few extra bucks your name
would add a little class to her company.

[Matt Mahoney]

I guess if anyone is foolish enough to buy the patent then they would be foolish enough to pay somebody up front to write the code. It wouldn't be hard to make up some test cases where it actually worked

[Alexander Rhatushnyak]

What is claimed is:

1. A method for handling a data stream having a number of data objects, comprising:

a step of selecting one or more numerical value as a compression base value;

a step of receiving the compression base value; and

a step of performing a compressing process on a data object based on a compression base value to obtain at least one compressed data result, wherein the data object is considered as one numerical value;

wherein the compressed data result is the direct result of mathematical operation on the data object using compression base value, wherein the compression process can be used to further compress the compressed data result.

I guess if the output of your RLE implementation is still RLE-compressible, the patent owner may sue you, and you'll have to prove that the byte your RLE implemetation outputs is not a compression base value, and the data it decompresses to is not a data object based on a compression base value.

[Matt Mahoney]

Well, you could always claim prior art on recursive compression. http://mattmahoney.net/dc/barf.html

[biject.bwts]

Matt I consider you a friend. So if you apply for a job with this lady you could reference these facts.

1) she is a strong supporter of the DNC
2) you are a friend of David Scott
3) David Scott is the democratic representative of the 13th district in Georgia
4) If she searches the web she would see much that confirms these facts


The above is all true. How can one disagree with that?

[Matt Mahoney]

http://davidscott.house.gov/
The resemblance is striking

[biject.bwts]

Yes it does look a lot like the picture of David Scott at http://www.squeezechart.com/people.html so I guess that
would be farther proof you could use in getting a job with the recursive compression lady.

[Matt Mahoney]

I dunno, looks a lot like this guy. http://www.apitalk.com/interviews/David-A-Scott.html

[giorgiotani]

It wouldn't be hard to make up some test cases where it actually worked

And for other cases, hiding data in temporary files would be enough to convince someone fool enough to buy that patent to accept the working implementation of the magic compressor!

[Matt Mahoney]

I guess there is no shortage of fools who believe in recursive compression.
http://www.nzherald.co.nz/technology...ectid=10665070

[Sportman]

I guess there is no shortage of fools who believe in recursive compression.
http://www.nzherald.co.nz/technology...ectid=10665070

I checked all law suite documents some years ago it was a 50/50 case, at least his first years it looked he had it working but if he lost it later and then faked it, he also spent as many inventors investors money at personal goals instead of making a profitable company.

There is an new kid on the block:

Dr Constant Tsai Shi Wong:
http://www.recursiveware.com/profile.html
http://au.linkedin.com/pub/constant-wong/15/609/325
http://www.facebook.com/people/Constant-Wong/1096964729

He claim a working random recursive compressor:

http://www.recursiveware.com/techinfo.html
http://www.recursiveware.com/faq.html
http://www.recursiveware.com

Mark Nelson about him:
http://www.drdobbs.com/blog/archives...igeonhole.html

[Matt Mahoney]

Yeah, that recursiveware might actually work since you have to tell the decompresser how many times to decompress. Here is one algorithm. To compress, treat the input file as a base 256 integer (with digits 1-256, not 0-255), and subtract 1. Repeat until you have 0 (empty file). To decompress, just reverse the process.

Edit: implemented as BARF2 (scroll to bottom and get barf2.zip). Took me a couple hours to solve a problem this guy has been working on since 2003. (GPL source and Windows .exe).

Plus, it's even bijective. I'm pretty sure that's a first for a recursive data compressor.

P.S. It is reasonably fast for files up to about 10 KB. Above that you can improve the speed by splitting the file into smaller pieces and compressing separately.

[biject.bwts]

Matt this is the real me. One of my kids found it. So it's the only real pic of me on the net. I feel pretty good lately. I think it's due to the German beer I stocked up on. Any way I plan to update some of the BWTS stuff. I changed UNBWTS today so that it's still linear in time and space and a true 5n in memory version. I don't have a large memory machine but will write a very simple test compressor using it. The avatar is me around 1972

[Omnikam]

Funny story that, and the money they want for it. Has anyone played around with changing different file extensions to .txt and then Compressing the .txt file? i have and can say that once the compressed text file is decompressed and its extension is changed back to the original one you still have a working file. But Working on converting to binary is simple and by the sounds of it brilliant. Talk about repeating numbers to recursively compress again and again you would probably just need a few different dictionary's for each layer of compression
What do you think?

Qin Zhang

i remember that name???
Always thought there was a way to covert any file to a string of binary number's then recursively compress the binary numbers. even the dictionary would be relatively small
Looked at BARF looks promising 10k file size
This might sound like a silly idea, but couldn't you rename any file extension to txt opening it up in note bad you get lots if ASCII gibberish, but you could then convert ASCII to binary and then Compress the Binary numbers as a text file. reverse the order and you have your file back. Just a thought, but please comment on it possibility's if any. Even better ill try it out as i have an ASCII 2 binary converter

Update: it works but the binary text file doesnt compress any better than the original file and is sometimes Larger. I was using a 32k zip file which i then changed the exe into .txt then used ASCII 2 binary and got a 440k text file, i think the best i was able to compress this file was to around 44k, but i think thats because it was such a small file and the Achive had to store its Dictionary which didnt help. A Large binary text file should compress better i would think?

v Thanks Matt ill be reading this for sure

[Matt Mahoney]

Well, some compressors use different models depending on file type, so I expect changing the extension to .txt might make compression worse unless it really was a text file.

Anyway, about recursive compression, see http://mattmahoney.net/dc/dce.html (section 1.1). To decompress an empty file with BARF2 you have to tell it how many times. However that number (written in decimal) is larger than the original file.

[Omnikam]

So what your saying then Matt is although it Works, in the end you still need a file with the All the decimal numbers to decompress it, and that file would be larger than that original file So is pointless?
Also after reading your Barf explanation it seems your taking the piss out of the ignorant, or maybe i'm just being ignorant :P
LOL Matt i used the cN method and entered a very large number and the output size was identical to the input? fishy
So i tried the c command and thought wow it work's until i saw the size of the barf2.count file, its bigger than the original file, but i guess thats what you have been saying all along. Impossible

Matt check this us patent out http://www.wikipatents.com/US-Patent...ing-text-files
What do you think?
Matt ,I was looking at the barf2.count file in notepad and all i see is numbers, but there are patterns in it.eg there where many 888 or 898 and many combination's of Patterns. What if i had a permanent list stored on the computer for repeating numbers over 2 digits long and give them a 1 digit Alphabetical representation For example 766543246675454 Would be broken down to 66=A 54=B and so on. 26 Alphabetical one digit representation in Lower Case and 26 in upper Case then use the @#$%^& symbols etc All representing a numerical pattern over 2 digits in length. Then Scan the file with all those numbers and compare it against the Static list and substitute those numerical patterns with the appropriate letter, being upper or lover case.
This would then reduce the size of the File by a little. Then when you want to convert back it only looks for the letters and replaces then with the number from the list, the list that never changes. So the order needn't be stored in the file As its a fixed equation. Im not a mathematician, but 26*2 is 52 different Alphabetical representations plus the Abstract symbols So maybe another 30. A nice total of 82 different 1 digit combination's representing a predefined list of over 2 digit numerical patterns. So the End result might look like 3A$g47;918hF and i know that A=66 and $=87 ect so rebuilding would be simple.But then i got thinking since im only comparing to a predefined list and since many off the numbers would be replaced with a letter or symbol there would be new numerical patterns! Anyway i guess im saying that im hooked on compression since discovering this site. Its 5am and ive been siting up looking at that barf2.count file and all its numbers, and thinking of ways to make it smaller without using complex algorithms of which i know not.hehe I hope i didnt make anyone cringe, but ill be back with my solution. And Ill read The Book Matt..goodnight
UPDATE: could have more than one static dictionary and the program scans to see which dictionary has the most matches and uses that one on the barf2.count file

[Matt Mahoney]

Looks like Microsoft's help file compression: RLE + dictionary + Huffman with a search index. Patent is expired now.

[Omnikam]

To decompress an empty file with BARF2 you have to tell it how many times. However that number (written in decimal) is larger than the original file.

Matt why does it have to be Stored as a decimal, couldnt the Very large decimal number be represented by a mathematical formula that is derived specifically from the decimal number. So you could then delete the number but still get it back using the formula? because then you would only have the size of the formula stored. which would be very small i think? Then the formula gets called and recreates the very large decimal number, i know that mean's the formula would have to be dependent on the actual decimal number, but isnt there a way to take any number no mater how large and unique and create a formula to get that number, i mean you already have the answer in the form of the large decimal number couldn't you juat reverse a formula based on the decimal number then store the formula, deleting the large decimal number, and when you need the decimal number again to decompress, you just use the formula to get it back? anyway that was just an idea that i had, not much sleep i guess, and a new love of numbers. but i did think instead of looking for patterns in the number.look at mathematical formulas to get the number back. Could you please offer me feed back on this idea, as it seems good 6am in the morning
Maybe continually square the number and until it's reduced in size but keep a record of how many times you squared it and the last square smallest squared number, store the number+times squared, reverse equation to give original number. Would that work?

[m^2]

Matt why does it have to be Stored as a decimal, couldnt the Very large decimal number be represented by a mathematical formula that is derived specifically from the decimal number. So you could then delete the number but still get it back using the formula? because then you would only have the size of the formula stored. which would be very small i think? Then the formula gets called and recreates the very large decimal number, i know that mean's the formula would have to be dependent on the actual decimal number, but isnt there a way to take any number no mater how large and unique and create a formula to get that number, i mean you already have the answer in the form of the large decimal number couldn't you juat reverse a formula based on the decimal number then store the formula, deleting the large decimal number, and when you need the decimal number again to decompress, you just use the formula to get it back? anyway that was just an idea that i had, not much sleep i guess, and a new love of numbers. but i did think instead of looking for patterns in the number.look at mathematical formulas to get the number back. Could you please offer me feed back on this idea, as it seems good 6am in the morning

I suggest that you read Matt's book, he answered all these questions already.
In short:
Yes, generating a function that generates the file (or the number, it's the same really) could work great. But finding the shortest possible function is not computable.
And even if it was, it wouldn't go around the pigeon hole theorem, it wouldn't reduce the size of any file.

[Matt Mahoney]

Most large numbers don't have small formulas that describe them, no matter how hard you look. This is a common fallacy among random data compression enthusiasts. They will give examples, like (3^10^10)/7 which produces a number with billions of digits. Such examples are easy to find. But the counting argument proves that among large numbers, those with simple formulas are extremely rare. This is the method described in the patent at the beginning of this thread, and it doesn't work. But it is useless to argue against them. Mostly they have a poor understanding of math so they will not understand your arguments. They will flounder for years in their attempts and never produce a working program. You can find lots of examples in comp.compression. I rarely visit it now because it has become a wasteland because all the people that write real compression software got tired of arguing with the fools and left. A few are con artists, but most genuinely believe that they can make their methods work.

If you want to compress barf2.count, then change the program to store the number in binary or hexadecimal instead of decimal. Then compress that file with a real compressor like zip or ppmonstr or something. The compression should be almost as good as if you just used that compressor in the first place. (It won't be any better).

Or do some research. Read my book. See what's been done, like http://mattmahoney.net/dc/text.html

[Omnikam]

Thanks Matt ill defiantly be reading your book, i guess what i suggested if like the holy grail of mathematics. And as elusive i have a copy of your book so i will do some reading
ps i was thinking along the lines of continuously squaring a large number until its reduced to it lowest possible(cannot be squared further),that number is then stored along with the number of times it was squared, because the number of times its squared is known it could be represented in 1000's 100's 00's an 0's there fore simplifying the equation back to its original number. But i agree in that i don't know enough about it. So ill read your book and study Fibonacci and the other greats, in hope i can understand more of this fascinating world.
UPDATE: what about with smaller numbers? say up to 5 digits long, could you cut the decimal numbers into 5 digit chunks and process a formula based on the smaller numbers, and then move to the next chunk and so forth until the entire decimal strand has been formulated? actually don't answer, ill read and learn so i can ask educated questions :P
But one last exited child's perhaps silly question, then i promise to study

Would it be possible to create a "learning archiving" program based on pattern recognition that grows the more patterns it's exposed too, similar to human memory. Im aware that the data base would grow massively, but then wouldn't the trade off eventually be compression based on remembered patterns and maybe associations. lol i haven't slept for 2 nights.
If i compare the contents of A to the contents of B where A is greater than B then i need only add the difference between A with B to A and of coarse A does become larger, but them Shouldn't in Time A become Large enough to recognize B in all its form's?

[Matt Mahoney]

A prediction model will learn, but it is a little different than the way humans learn. To decompress you have to reset the model to its initial state and repeat the learning process exactly from the data as it decompresses. That is hard to do with human brains

But you could also use compression to transmit small updates to files. You train a model on the old file, then compress the new file, which should be much smaller than if it was compressed by itself. If the receiver has the old file, then the decompresser would train the model on it, then decompress the new file. (I am considering adding this feature to ZPAQ).