Results 1 to 11 of 11

Thread: UDA 0.303

  1. #1
    Member
    Join Date
    May 2012
    Location
    United States
    Posts
    342
    Thanks
    197
    Thanked 58 Times in 42 Posts

    UDA 0.303

    UDA 0.303.

    Full disclosure: I have no formal programming or data compression education. I always liked UDA 0.301 and when I looked at the source code, it seemed fairly understandable...So I started playing around.

    I released UDA 0.302 for benchmarking purposes only for the SqueezeChart. After seeing the improvement in compression ratio and some interest from members on this forum, I decided to keep playing around.

    UDA 0.303, unlike all previous releases, has 3 version:

    MAX
    This one is tuned for absolute maximum compression ratio-speed is not important. This version should have a better compression ratio than 0.302.

    FAST
    The goal of this version is to achieve almost identical compression ratios as UDA 0.301 released by dwing almost 10 years ago by increasing the memory usage and making other modifications to the code.

    VERYFAST
    This is the speed king! The goal was to achieve the fastest possible speeds while still keeping a good compression ratio. Some modifications to the source code would increase the speed by 200% but compression ratio would drop by 200% also-so that was unacceptable.

    All versions require 2GB memory and are 32-bit Windows executables compiled with the fastest possible settings using Intel C++ 11.1 compiler.

    Looking forward to seeing benchmark results and hearing opinions!

    EDIT Sources have been added. Please note that after compiling, EDITBIN /LARGEADDRESSAWARE must be used or a memory error will occur.
    Attached Files Attached Files
    Last edited by comp1; 1st October 2015 at 15:16.

  2. Thanks (7):

    Bulat Ziganshin (29th September 2015),Hacker (29th January 2017),lorents17 (29th September 2015),Minimum (29th September 2015),nikkho (30th September 2015),samsat1024 (3rd October 2015),SvenBent (3rd October 2015)

  3. #2
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,572
    Thanks
    780
    Thanked 687 Times in 372 Posts
    PAQ is GPL, so you should publish sources too. at least until you will start saying that you have developed everything from the scratch

  4. Thanks:

    nikkho (30th September 2015)

  5. #3
    Member Skymmer's Avatar
    Join Date
    Mar 2009
    Location
    Russia
    Posts
    688
    Thanks
    41
    Thanked 173 Times in 88 Posts
    comp1, your wish for glory has blinded your mind.
    You have 48 hours to confess and to publicly tell the right story of your "development". Otherwise I will be forced to do it myself and I can't guarantee the civilized and polite storytelling.
    One modest "reminder" is in the attachment below.

    Click image for larger version. 

Name:	scr_20150930_220313.png 
Views:	264 
Size:	21.5 KB 
ID:	3858

  6. #4
    Member
    Join Date
    May 2012
    Location
    United States
    Posts
    342
    Thanks
    197
    Thanked 58 Times in 42 Posts
    Quote Originally Posted by Skymmer View Post
    comp1, your wish for glory has blinded your mind.
    You have 48 hours to confess and to publicly tell the right story of your "development". Otherwise I will be forced to do it myself and I can't guarantee the civilized and polite storytelling.
    One modest "reminder" is in the attachment below.

    Click image for larger version. 

Name:	scr_20150930_220313.png 
Views:	264 
Size:	21.5 KB 
ID:	3858


    Skymmer Yes, you did initially help me with this. What you did for me did not work though. Your adjustment did increase the memory slightly but I further tweaked the portion of the code that you suggested to increase it further.

    So, to the board: Yes, Skymmer was very influential in helping me begin my journey of modifying the source. His help led me to continue further adjusting the portion of the code he mentioned along with the rest of the code.

    A clarification so nobody gets the wrong idea: I am NOT a programmer, scientist, etc. (as mentioned in the first post). I do not have any intention of taking full credit or "glory" for this.

    I do not want to start any trouble folks. I was helped initially by Skymmer. After this point, the many other modifications to the code were from me.

    Thank you Skymmer for all of your help at the beginning. If it weren't for your helpful response to my question in that PM you posted an image of, these 3 versions of UDA would not have been created because I would have not known where to begin.

    PS: Original attachment updated in first post to include sources.
    Last edited by comp1; 1st October 2015 at 15:11.

  7. Thanks (2):

    Bulat Ziganshin (1st October 2015),ne0n (2nd October 2015)

  8. #5
    Member Skymmer's Avatar
    Join Date
    Mar 2009
    Location
    Russia
    Posts
    688
    Thanks
    41
    Thanked 173 Times in 88 Posts
    My wish for glory is satisfied now

    And now seriously. I really thought that you're weaseling behind my back because giving compile to Stephan only and then saying that you lost the sources looks quite suspicious. I even thought that you simply modified my compile in HEX and send it to Stephan.
    Somebody may think that I just forced you to mention my person to satisfy my complacency but the real reason is the following. I personally think that the honesty and transparent relations are the only way to build and to keep the good community as we got here. Even if it means a harshness sometimes.

    I'm glad that you step into the right direction, comp1.

  9. Thanks:

    comp1 (2nd October 2015)

  10. #6
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    510
    Thanks
    208
    Thanked 348 Times in 185 Posts
    comp1, you can make it little faster. Replace squash with precalculated version, latest paq8 or zpaq has one. And dot_product, train with SSE2 versions. And also ContextMap::E::get can be replaced with SSE2 version.
    KZo


  11. #7
    Member Skymmer's Avatar
    Join Date
    Mar 2009
    Location
    Russia
    Posts
    688
    Thanks
    41
    Thanked 173 Times in 88 Posts
    Quote Originally Posted by comp1 View Post
    Please note that after compiling, EDITBIN /LARGEADDRESSAWARE must be used or a memory error will occur.
    Well, you can avoid that and make the things more simple.

    In case of Intel Compiler just use: /link /LARGEADDRESSAWARE
    Please note that /link options implies that all the options after it are for linker.

    In case of GCC\MingW just use: -Wl,--large-address-aware

  12. #8
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    925
    Thanks
    57
    Thanked 116 Times in 93 Posts
    Quote Originally Posted by Bulat Ziganshin View Post
    PAQ is GPL, so you should publish sources too. at least until you will start saying that you have developed everything from the scratch
    I believe it was from "crash" if I recall the correct wording. So clearly what was meant, was he poked around in the source at random, and whatever didn't crash, he tested for improvements or not.

  13. #9
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,572
    Thanks
    780
    Thanked 687 Times in 372 Posts
    Sven, i meant words that hamidi said about likeness of lzturbo and tornado

  14. #10
    Member
    Join Date
    May 2012
    Location
    United States
    Posts
    342
    Thanks
    197
    Thanked 58 Times in 42 Posts
    Quote Originally Posted by kaitz View Post
    comp1, you can make it little faster. Replace squash with precalculated version, latest paq8 or zpaq has one. And dot_product, train with SSE2 versions. And also ContextMap::E::get can be replaced with SSE2 version.
    Thanks but now you're getting a little too advanced for me!

    I'm hoping to see benchmarks from Stephen, Matt, etc. Now that I made the source available as Bulat suggested, others are welcome to make improvements and post them here. I always thought UDA had potential...I also knew that others more qualified than I would find more possibilities for improvement.

    Also, thank you Skymmer for the /link switch for the compiler...Can't believe I missed that when I spent a lot of time reading and testing different options to create the fastest compiles.

    Thanks again everyone.

  15. #11
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    925
    Thanks
    57
    Thanked 116 Times in 93 Posts
    Quote Originally Posted by Bulat Ziganshin View Post
    Sven, i meant words that hamidi said about likeness of lzturbo and tornado
    Yeah i know. I was just referring to his typo, that he actually wrote he made it from "Crash" and not "Scratch". I might recall incorrectly. For some reason it made me imagine he (Hamid, Do Not Dissturb, DND, rafikhan?) was just poking randomly in the source of tornado without really knowing what he was doing and just was lucky with his results.... an infinite monkey/typewriter thing

    I remember the issues with the identical "UI", even having the same 3trd option not available, but present in the UI. The identical prime numbers. The identical list of file extension even with ... (ermm was it mat's? i think it was) addition to the file list. and some very ... political way of answering things.


    on topic: Sadly my weekend is all busy, but i might test the new UDA out soons i remember it gave some pretty decent results back in the days.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •