UDA 0.303

[comp1]

UDA 0.303.

Full disclosure: I have no formal programming or data compression education. I always liked UDA 0.301 and when I looked at the source code, it seemed fairly understandable...So I started playing around.

I released UDA 0.302 for benchmarking purposes only for the SqueezeChart. After seeing the improvement in compression ratio and some interest from members on this forum, I decided to keep playing around.

UDA 0.303, unlike all previous releases, has 3 version:

MAX
This one is tuned for absolute maximum compression ratio-speed is not important. This version should have a better compression ratio than 0.302.

FAST
The goal of this version is to achieve almost identical compression ratios as UDA 0.301 released by dwing almost 10 years ago by increasing the memory usage and making other modifications to the code.

VERYFAST
This is the speed king! The goal was to achieve the fastest possible speeds while still keeping a good compression ratio. Some modifications to the source code would increase the speed by 200% but compression ratio would drop by 200% also-so that was unacceptable.

All versions require 2GB memory and are 32-bit Windows executables compiled with the fastest possible settings using Intel C++ 11.1 compiler.

Looking forward to seeing benchmark results and hearing opinions!

EDIT Sources have been added. Please note that after compiling, EDITBIN /LARGEADDRESSAWARE must be used or a memory error will occur.

[Bulat Ziganshin]

PAQ is GPL, so you should publish sources too. at least until you will start saying that you have developed everything from the scratch

[Skymmer]

comp1, your wish for glory has blinded your mind.
You have 48 hours to confess and to publicly tell the right story of your "development". Otherwise I will be forced to do it myself and I can't guarantee the civilized and polite storytelling.
One modest "reminder" is in the attachment below.

[comp1]

comp1, your wish for glory has blinded your mind.
You have 48 hours to confess and to publicly tell the right story of your "development". Otherwise I will be forced to do it myself and I can't guarantee the civilized and polite storytelling.
One modest "reminder" is in the attachment below.

Skymmer Yes, you did initially help me with this. What you did for me did not work though. Your adjustment did increase the memory slightly but I further tweaked the portion of the code that you suggested to increase it further.

So, to the board: Yes, Skymmer was very influential in helping me begin my journey of modifying the source. His help led me to continue further adjusting the portion of the code he mentioned along with the rest of the code.

A clarification so nobody gets the wrong idea: I am NOT a programmer, scientist, etc. (as mentioned in the first post). I do not have any intention of taking full credit or "glory" for this.

I do not want to start any trouble folks. I was helped initially by Skymmer. After this point, the many other modifications to the code were from me.

Thank you Skymmer for all of your help at the beginning. If it weren't for your helpful response to my question in that PM you posted an image of, these 3 versions of UDA would not have been created because I would have not known where to begin.

PS: Original attachment updated in first post to include sources.

[Skymmer]

My wish for glory is satisfied now

And now seriously. I really thought that you're weaseling behind my back because giving compile to Stephan only and then saying that you lost the sources looks quite suspicious. I even thought that you simply modified my compile in HEX and send it to Stephan.
Somebody may think that I just forced you to mention my person to satisfy my complacency but the real reason is the following. I personally think that the honesty and transparent relations are the only way to build and to keep the good community as we got here. Even if it means a harshness sometimes.

I'm glad that you step into the right direction, comp1.

[kaitz]

comp1, you can make it little faster. Replace squash with precalculated version, latest paq8 or zpaq has one. And dot_product, train with SSE2 versions. And also ContextMap::E::get can be replaced with SSE2 version.

[Skymmer]

Please note that after compiling, EDITBIN /LARGEADDRESSAWARE must be used or a memory error will occur.

Well, you can avoid that and make the things more simple.

In case of Intel Compiler just use: /link /LARGEADDRESSAWARE
Please note that /link options implies that all the options after it are for linker.

In case of GCC\MingW just use: -Wl,--large-address-aware

[SvenBent]

PAQ is GPL, so you should publish sources too. at least until you will start saying that you have developed everything from the scratch

I believe it was from "crash" if I recall the correct wording. So clearly what was meant, was he poked around in the source at random, and whatever didn't crash, he tested for improvements or not.

[Bulat Ziganshin]

Sven, i meant words that hamidi said about likeness of lzturbo and tornado

[comp1]

comp1, you can make it little faster. Replace squash with precalculated version, latest paq8 or zpaq has one. And dot_product, train with SSE2 versions. And also ContextMap::E::get can be replaced with SSE2 version.

Thanks but now you're getting a little too advanced for me!

I'm hoping to see benchmarks from Stephen, Matt, etc. Now that I made the source available as Bulat suggested, others are welcome to make improvements and post them here. I always thought UDA had potential...I also knew that others more qualified than I would find more possibilities for improvement.

Also, thank you Skymmer for the /link switch for the compiler...Can't believe I missed that when I spent a lot of time reading and testing different options to create the fastest compiles.

Thanks again everyone.

[SvenBent]

Sven, i meant words that hamidi said about likeness of lzturbo and tornado

Yeah i know. I was just referring to his typo, that he actually wrote he made it from "Crash" and not "Scratch". I might recall incorrectly. For some reason it made me imagine he (Hamid, Do Not Dissturb, DND, rafikhan?) was just poking randomly in the source of tornado without really knowing what he was doing and just was lucky with his results.... an infinite monkey/typewriter thing

I remember the issues with the identical "UI", even having the same 3trd option not available, but present in the UI. The identical prime numbers. The identical list of file extension even with ... (ermm was it mat's? i think it was) addition to the file list. and some very ... political way of answering things.


on topic: Sadly my weekend is all busy, but i might test the new UDA out soons i remember it gave some pretty decent results back in the days.