Page 1 of 2 12 LastLast
Results 1 to 30 of 57

Thread: quad 1.02a has been released!

  1. #1
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    Okay! A new version featuring improved exe compresison is here!

    Enjoy!

    Link:
    quad102a.zip (26 KB)


  2. #2
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    Thanks! At least something to break today's depression...

  3. #3
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by encode
    Okay! A new version featuring improved exe compresison is here!

    Enjoy!
    Thanks Ilia!

    Quote Originally Posted by Black_Fox
    Thanks! At least something to break todays depression...
    Ditto!

  4. #4
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    Quad 1.02a improves on its predecessor by 170,475 bytes in my test.
    Some files were inflated by few bytes (XM, MP3, PDF, PSD, BMP, PPS, SAVE).
    Performance on EXE and DLL files respectively:
    original - 2,232,320 - 3,432,448
    quad 1.01a - 693,331 - 1,519,479
    quad 1.02a - 624,926 - 1,410,564

  5. #5
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    Quote Originally Posted by Black_Fox
    Some files were inflated by few bytes (XM, MP3, PDF, PSD, BMP, PPS, SAVE).
    Possibly, all things with auto detection hurts compression is some cases. This exe one can produce tiny inflate on some binary data, on ASCII text files this exe filter has no effect.


  6. #6
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    You're true, it's better with auto anyway. I like simplicity of your compressors' UI - only e|d, infile, outfile parameters.
    Could it be possible to look after filetype headers, so additional filtering is applied only on files exe-filter is made for?

    EDIT:fixed grammar.

  7. #7
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    Quote Originally Posted by Black_Fox
    Could it be possible to look after filetype headers, so additional filtering is applied only on files exe-filter is made for?
    It is possible. For example, PIM archiver looks for file header and if needed enables a special fitler.

    For EXE, DLL, OCX files we can look for "MZ" at the beginning. Cirrently I will keep auto, since this approach is effective for TAR and other files with x86 code (Linux ELF executables, etc.).

    Also I have an idea about to add some simple header to the QUAD file.

    Current QUAD file header:

    <uncompressed size>, 4 bytes
    <compressed data>, X bytes

    As the example, improved header:

    "quad" magic, 4 bytes
    flags or transformType, 1 byte
    <uncompressed size>, 4 bytes
    <compressed data>, X bytes

    With flags we can define anything, including:

    CRC32 included
    Date/Time included
    FileName included
    Filters used:
    x86transform();
    tabtransform();
    etc.
    And extra reserved flag for future header extensions.

    I we dont plan any additional features inside QUAD files, we can define just "transformType" field to enable or disable any transformations. For example, the x86 transformer can be enabled by default - since it uses auto detection, but if user wish, user can disable it. But we must have a lots of filters first...


  8. #8
    Guest
    Tested on http://uclc.info

    Back in business

  9. #9
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    Quote Originally Posted by Johan
    Thank you Johan!

  10. #10
    Guest
    My Test:
    143 EXE files in one tar-file 327 Mb (largest file - 18 Mb, smallest - 0.5 Mb; from my "crogram files") :
    NAME - TIME compress/decompress - Result size:
    quad 1.02a - ( compress in 21 min 25 sec, decompress in 2 min 14 sec ) - 138 Mb
    WinRAR 3.62 - (max 4Mb solid) - ( compress in 6 min 17 sec, decompress in 38 sec) - 128 Mb
    7-zip 4.43b - (Ultra 32 Mb solid) - ( compress in 9 min 00 sec , decompress in 49 sec ) - 110 Mb
    WinZip 11 - (Optim. for best compr.) - ( compress in 6 min 55 sec , decompress in 6 min 51 sec ) - 140 Mb
    Pim 1.25b - ( compress in 6 min 03 sec , decompress in 5 min 13 sec ) - 143 Mb

  11. #11
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    Huh, at least it is not a flawless victory from the world's best compressors! Quad is holding punch! If seriously:

    WinZip 11 – is a lame usage of the stolen PPMd compression engine.

    WinRAR additionally to the E8/E9 transformer has stuff like table/delta transformer, which can make the difference.

    7-Zip is King of the World!!! Nuff said!

    Well, mess with the best die like the rest... But if you compare Quad to others, keeping in mind memory usage/decompression speed (and to be honest the size of source code, which is even about a few times smaller compared to the LZMA decoder...)
    ___
    Never fear!
    I is here!

  12. #12
    Guest
    Test for TC 5.1 dev7 - ( compress in 31 min 16 sec , decompress in 7 min 45 sec ) - 120 Mb.

  13. #13
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    Note on Quad. Currently, I use a brute-force approach to get the best possible compression from this engine. For example, if I change Flexible Parsing (Max mode) to the Lazy Evaluation (Normal mode), compression will be about 4x-8x or even more times faster with really tiny compression loss. (I'm just focusing on decompression speed only)


  14. #14
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by encode
    WinZip 11 – is a lame usage of the stolen PPMd compression engine.
    Dont forget the stolen Wavpack!

    Quote Originally Posted by encode
    7-Zip is King of the World!!! Nuff said!
    I nearly fell off my chair when I saw this! I understand why you wrote it though.

    Quad is showing very good performance even at this early stage in its development. It should be a truly awesome compressor when the final is released!

  15. #15
    Guest
    And for Pimple 1.43b result is:
    Pimple 1.43b - (Maximum 128 Mb) - ( compress in 33 min 56 sec , decompress in min sec ) - 129 Mb.
    I hope Quad will be improved in final release.
    Good luck

  16. #16
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    Currently Quad stays in active development stage. Allso I'm thinking about:

    1. Improved LZ output coding - i.e. to eliminate the "A10.jpg" problem. Since here we use arithmetic coder, with just slightly improved model we will be able to compress this JPEG file. To do so we must use some kind of SSE technique, probably like in LZPXJ:
    encode(freqUnder + (s * h), freq + h, totFreq + (N * h));
    Anyway, currently I'm consulting with some people about that.

    2. Table and/or Delta preprocessor, of course with auto detection. I will try to find a fast and simple solution that able to improve compression of EXE files. For example, try to Enable/Disable some options before compressing files with WinRAR:
    Advanced tab
    Compression... button
    Enable 32-bit executable (Pentium) compression
    Enable Delta compression
    and see how compression changes.


  17. #17
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    How about adding a feature that stores a file in the archive without compression if the compressed file size is bigger than original?

  18. #18
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    Quote Originally Posted by Black_Fox
    How about adding a feature that stores a file in the archive without compression if the compressed file size is bigger than original?
    Currently, I am working on the compression engine. The storing files without a compression are not a compression related trick. Moreover, what if a TAR archive will already contain a compressed file somewhere in the middle?

    So anyways, currently I am looking into the best PPM sources (PPMD+, PPMZ, PPMd, ...) to find some additional modeling tricks and ideas...


  19. #19
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    Quote Originally Posted by encode
    Moreover, what if a TAR archive will already contain a compressed file somewhere in the middle?
    it wont find already compressed part in the middle of the file, but if the tar archive contents were already compressed entirely, you could get some kBs out of it...
    edit: or maybe you could add some precomp routines... when the source code gets available

  20. #20
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    Of course, it will be better if the engine will be more stable on already compressed data...

  21. #21
    Guest
    How about adding small GUI for Quad? Like Dark or better...

  22. #22
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    Quote Originally Posted by Squxe
    How about adding small GUI for Quad? Like Dark or better...
    Currently, I am focused on compression engine only. For example, I am experimenting with "Full Exclusions" – a well-known trick for the PPM. Note the most PPM coders (PPMd, PPMZ, HA) uses such trick. With this feature, Quad shows some improvements on ALL files. On text files, the difference is small, and on binary and random data, we can find a nice improvement.

    The GUI and adding Quad support for PIM is somewhere far in future...


  23. #23
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    Some testing results:

    Testing files:
    three files form the SFC:
    A10.jpg - as a hard-to-compress file
    acrord32.exe - as a binary file
    world95.txt - as a text file

    Modifications of Quad:
    Quad 1.02a - current public version
    Quad 1.03a-fb - uses full blending
    Quad 1.03a-fx,x1.0 - uses full exclusions, model update speed x1.0
    Quad 1.03a-fx,x1.5 - uses full exclusions, model update speed x1.5
    Quad 1.03a-fx,x2.0 - uses full exclusions, model update speed x2.0

    A10.jpg
    Quad 1.03a-fb: 831,899 bytes
    Quad 1.03a-fx,x1.0: 841,869 bytes
    Original: 842,468 bytes
    Quad 1.03a-fx,x1.5: 844,833 bytes
    Quad 1.03a-fx,x2.0: 848,317 bytes
    Quad 1.02a: 855,819 bytes

    acrord32.exe
    Quad 1.03a-fb: 1,505,736 bytes
    Quad 1.03a-fx,x2.0: 1,511,596 bytes
    Quad 1.03a-fx,x1.5: 1,515,523 bytes
    Quad 1.03a-fx,x1.0: 1,523,136 bytes
    Quad 1.02a: 1,524,108 bytes
    Original: 3,870,784 bytes

    world95.txt
    Quad 1.03a-fx,x1.0: 611,203 bytes
    Quad 1.03a-fx,x1.5: 612,146 bytes
    Quad 1.02a: 613,119 bytes
    Quad 1.03a-fx,x2.0: 613,470 bytes
    Quad 1.03a-fb: 618,052 bytes
    Original: 2,988,578 bytes

    I think the best one is the "Quad 1.03a-fx,x1.5". Quad with full blending becomes noticeable slower, update speed x1.0 is badly performs on binary data, and x2.0 performs badly on textual data. So, The x1.5 is in the middle. Also, if at all, the speed of Quad with Full Exclusions is not affected and in ALL cases this trick gives compression gain (especially on small and/or hard compressible and binary data).

    Additional tests:

    worms2.tar - Worms 2 Demo game
    Quad 1.03a-fb: 8,760,794 bytes
    Quad 1.03a-fx,x1.5: 8,872,624 bytes
    Quad 1.02a: 8,884,476 bytes
    Original: 17,097,728 bytes

    mptrack.exe - ModPlug Tracker software
    Quad 1.03a-fx,x1.5: 511,209 bytes
    Quad 1.03a-fb: 514,184 bytes
    Quad 1.02a: 516,120 bytes
    Original: 1,159,172 bytes


  24. #24
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    How much memory does Quad require now?

  25. #25
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    Quote Originally Posted by LovePimple
    How much memory does Quad require now?
    Exactly the same amount!

  26. #26
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by encode
    Exactly the same amount
    Excellent! Results are impressive!

  27. #27
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    A baseline PPM has potential codespace wastage. This new trick eliminates such problem. Note with higher order PPM such trick is able to kill the speed, but at low orders, it is possible to keep the same speed. Also, note the speed is slightly "affected" (negligible milliseconds) only on small files, with large files speed is the same. In addition, the compression improvement is not depended from the file size and usually it leads to the ~10...15 KB at max. But since we don't loose the speed why not to include such trick?! Anyway, I will continue experimenting...


  28. #28
    Guest
    hi
    I have download redline.tar.quad but i am not able to extract it please tellme how i extract this file
    thanx

  29. #29
    Tester

    Join Date
    May 2008
    Location
    St-Petersburg, Russia
    Posts
    182
    Thanks
    3
    Thanked 0 Times in 0 Posts
    How about public release of quad 1.03a (for testing purposes)?

  30. #30
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    Quote Originally Posted by Anonymous
    hi
    I have download redline.tar.quad but i am not able to extract it please tellme how i extract this file
    thanx
    Simply follow next steps:
    1. Download the FIRST version of Quad (1.01a)
    2. Unzip quad.exe to some folder, for example to drive C:
    3. Place the redline.tar.quad to the same place as quad.exe
    4. Run the Command Prompt. (Start->Accessories->Command Prompt)
    5. Change the default directory (Cocuments and SettingsUser>) to the Quad one. To do so, type:

    Cocuments and SettingsUser>cd ..
    Cocuments and Settings>cd ..
    C:>

    6. Now, to unpack the Quad file, type this command:

    C:>quad d redline.tar.quad redline.tar

    7. After unpacking finished, use your favorite archiver to unpack the TAR file!

    Hope this helps!


Page 1 of 2 12 LastLast

Similar Threads

  1. QUAD 1.10 has been released!
    By encode in forum Forum Archive
    Replies: 10
    Last Post: 22nd March 2007, 12:40
  2. QUAD 1.08 has been released!
    By encode in forum Forum Archive
    Replies: 28
    Last Post: 16th March 2007, 12:17
  3. Quad 1.04a has been released! [!]
    By encode in forum Forum Archive
    Replies: 25
    Last Post: 15th February 2007, 19:01
  4. quad 1.03a has been released!
    By encode in forum Forum Archive
    Replies: 23
    Last Post: 7th February 2007, 21:23
  5. quad 1.01a has been released!
    By encode in forum Forum Archive
    Replies: 25
    Last Post: 1st January 2007, 22:50

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •