Page 1 of 2 12 LastLast
Results 1 to 30 of 37

Thread: LPREPAQ released!

  1. #1
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    From the MC guestbook...

    Quote Originally Posted by Christian Schneider
    Great news!

    I just uploaded lprepaq to my site. It combines lpaq1 by Matt Mahoney and Precomp v0.3.5 to a complete file compressor/decompressor.

    Please report if you find any bugs since it is the first version. Source (just for lprepaq, Precomp is inside a DLL) is included.

    Have a look at http://schnaader.info

    Greetings,
    Christian "schnaader" Schneider

  2. #2
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    Thanks LovePimple! Seems like competition for FreeArc has just began

  3. #3
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quick (7) test...

    A10.jpg > 699,694
    AcroRd32.exe > 1,337,906
    english.dic > 462,145
    FlashMX.pdf > 2,336,792
    FP.LOG > 402,877
    MSO97.DLL > 1,690,034
    ohs.doc > 740,634
    rafale.bmp > 728,506
    vcfiu.hlp > 514,071
    world95.txt > 439,357

    Total = 9,352,016 bytes

    This is a truly AWESOME compressor!!!


    Quote Originally Posted by Black_Fox
    Thanks LovePimple! Seems like competition for FreeArc has just began

  4. #4
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    Craziest!

  5. #5
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    Third place in SFC @ hundreds of kilobytes per second? Crazy!

  6. #6
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    lprepaq can't be a legitimate competition becuase it violates GPL license of lpaq

  7. #7
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Another ( quick test...

    A10.jpg > 699,694
    AcroRd32.exe > 1,335,985
    english.dic > 461,798
    FlashMX.pdf > 2,329,857
    FP.LOG > 402,393
    MSO97.DLL > 1,687,335
    ohs.doc > 740,378
    rafale.bmp > 728,270
    vcfiu.hlp > 513,286
    world95.txt > 438,541

    Total = 9,337,537 bytes


    ENWIK8 > 19,797,006 bytes

    SUPER-COMPRESSOR!!!

  8. #8
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by Black_Fox
    Third place in SFC @ hundreds of kilobytes per second? Crazy!
    Third place in SFC, very fast compression time, and still lots of room for tuning!!!

    Imagine PRECOMP + PAQ8L compression.

  9. #9
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    Quote Originally Posted by LovePimple
    Imagine PRECOMP + PAQ8L compression
    yes!

  10. #10
    Member
    Join Date
    Jul 2008
    Posts
    54
    Thanks
    0
    Thanked 0 Times in 0 Posts
    nice, really, nice, but (always a but...) lpaq1 still really slow in a p4 2.6gHz
    And program needs the dll packJPG22.dll

    What about to "inject" or use 7-zip compression?

  11. #11
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    Quote Originally Posted by John
    What about to "inject" or use 7-zip compression?
    wait for freearc 0.40

  12. #12
    Tester

    Join Date
    May 2008
    Location
    St-Petersburg, Russia
    Posts
    182
    Thanks
    3
    Thanked 0 Times in 0 Posts
    And program needs the dll packJPG22.dll
    Where it must be placed? I can't compress my jpg files
    edit:
    oopps... a10.jpg succesfully compressed. So Lprepaq can't compress all jpg files which can be compressed by PackJPG 2.2

  13. #13
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    First person to combine the power of PRECOMP with awesome compression of PAQ8M will become forever famous!

  14. #14
    Member Vacon's Avatar
    Join Date
    May 2008
    Location
    Germany
    Posts
    523
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Hello everyone,

    Quote Originally Posted by Bulat Ziganshin
    wait for freearc 0.40
    At least _I_ do!

    Quote Originally Posted by LovePimple
    First person to combine the power of PRECOMP with awesome compression of PAQ8M will become forever famous!
    Yep! And if he tunes speed of PAQ-part even more.
    But the race is opened.

    Best regards!

  15. #15
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    on my testset:

    _______name______________size________in___out speed
    FreeArc 0.40 -max ________11 842 821___733 / 1 057 (kB/s)
    LPREPAQ 1.0 -7 _________11 454 187___ 83 / ___92
    PAQ8M -7 + Precomp0.3.4__9 176 121_____3 / ____3

    I will try to provide details in a few days' time...

  16. #16
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by Vacon
    Yep! And if he tunes speed of PAQ-part even more.
    But the race is opened.
    Highest compression on the planet!!!

    Sorry! Just drooling!

  17. #17
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    Quote Originally Posted by Black_Fox
    I will try to provide details in a few days time...
    lprepaq (lpaq1) is not optimized for binary files

  18. #18
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 779 Times in 486 Posts
    I commented on lprepaq in the maximumcompression guestbook:
    http://www.maximumcompression.com/guestbook/gb.php

    But then I discovered that lprepaq doesn't compress some JPEG files that PackJPG 2.2 does, even though they are based on the same code. These 3 images: http://compression.ca/act/act-jpeg.html

    Almost all of the gain of lprepaq over lpaq1 is for flashmx.pdf (1.2 MB). Also 140K for a10.jpg. It does not seem to detect the large JPEG in ohs.doc, which would probably gain another 150K. I think it could be #1 with some x86 models (e8e9 filter), some binary models (sparse order 1-2), and a bmp model.

  19. #19
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by Matt Mahoney
    I think it could be #1 with some x86 models (e8e9 filter), some binary models (sparse order 1-2), and a bmp model.
    I hope someone capable is taking note!

  20. #20
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    LPREPAQ v1.1 has been released.

    Quote Originally Posted by Christian Schneider
    This improves JPG detection for JPG files with multiple SOI markers.

  21. #21
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quick ( test...

    A10.jpg > 699,694
    AcroRd32.exe > 1,335,983
    english.dic > 461,797
    FlashMX.pdf > 2,329,859
    FP.LOG > 402,393
    MSO97.DLL > 1,687,334
    ohs.doc > 740,378
    rafale.bmp > 728,270
    vcfiu.hlp > 513,283
    world95.txt > 438,541

    Total = 9,337,532 bytes

  22. #22
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    561
    Thanks
    212
    Thanked 200 Times in 93 Posts
    I just uploaded lprepaq v1.2.

    Changes:
    - Added Precomp switches to lprepaq.
    - Fixed lprepaq crashes.

    Have a look at http://schnaader.info
    http://schnaader.info
    Damn kids. They're all alike.

  23. #23
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Thanks Christian!

  24. #24
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    561
    Thanks
    212
    Thanked 200 Times in 93 Posts
    I just got a bug report that Precomp crashes under Windows Vista.
    So there could be some incompatibility issue for Windows Vista.
    Has anybody encountered the same for Precomp or lprepaq or managed running them under Vista?

    The bug report was for 32-bit Vista by the way, I don't know if that matters...
    http://schnaader.info
    Damn kids. They're all alike.

  25. #25
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 779 Times in 486 Posts
    lprepaq 1.2 seems to be twice as slow as lpaq1 on the large text benchmark. Was it compiled without optimization? The preprocessing step is very fast.

    http://cs.fit.edu/~mmahoney/compression/text.html# 1600

    Also, there is a new version, lpaq3, plus a version optimized for large text files, elpaq3 (from same source code), both by Alexander Rhatushnyak. These are compiled with g++. He sent me an Intel compile I haven't tested yet.

  26. #26
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 779 Times in 486 Posts
    Quote Originally Posted by Bulat Ziganshin
    lprepaq cant be a legitimate competition becuase it violates GPL license of lpaq
    No, it is legal. It includes source for the modified parts of lpaq1 (calling precomp to a temporary file). precomp is not under GPL and doesnt use any code from lpaq1.

  27. #27
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    561
    Thanks
    212
    Thanked 200 Times in 93 Posts
    Quote Originally Posted by Matt Mahoney
    lprepaq 1.2 seems to be twice as slow as lpaq1 on the large text benchmark. Was it compiled without optimization? The preprocessing step is very fast.
    Optimizations were enabled. G++ optimizations could be better than those of MSVC, but I doubt this.
    I guess it is the ((ftell(fin) % 3276 == 0) check I use in the main compress/decompress loops. Should be something like act_fin_pos and act_fin_pos++ instead of a ftell after every byte read in ...
    Ill compile a corrected version quickly.

    Quote Originally Posted by Matt Mahoney
    Also, there is a new version, lpaq3, plus a version optimized for large text files, elpaq3 (from same source code), both by Alexander Rhatushnyak. These are compiled with g++. He sent me an Intel compile I havent tested yet.
    Ill have a look at it and expect to release a version 1.3 that combines lpaq3, Precomp 0.3.6 (or perhaps upcoming 0.3.7? ) and packJPG 2.3 DLL (as soon as I receive it from Matthias Stirner).

    Quote Originally Posted by Matt Mahoney
    No, it is legal. It includes source for the modified parts of lpaq1 (calling precomp to a temporary file). precomp is not under GPL and doesnt use any code from lpaq1.
    Thanks for clarification. I assumed it would be that way, but wasnt sure because Im not that familiar with license stuff.
    Hopefully Ill get my code cleaned up soon and put both Precomp and the Precomp DLL that will be released at this point under LGPL.
    http://schnaader.info
    Damn kids. They're all alike.

  28. #28
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    Quote Originally Posted by schnaader
    Hopefully Ill get my code cleaned up soon and put both Precomp and the Precomp DLL that will be released at this point under LGPL.
    it will be great day. i wait imaptiently for a day when i will be able to integrate all these great libs into freearc

    i have some suggestions for precomp:

    1) use multiple threads to check various zlib compression settings. precomp should be an ideal multithreaded program, after all

    2) check first the most probable ones - last used, typically used (such as -6/-9 settings) and check in (very) fast mode only these typical settings

  29. #29
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    561
    Thanks
    212
    Thanked 200 Times in 93 Posts
    Quote Originally Posted by Bulat Ziganshin
    1) use multiple threads to check various zlib compression settings. precomp should be an ideal multithreaded program, after all
    Thats on my todo-list, but the way I use (de)flate routines is not optimal, so this will be changed first. These multi-thread/core specific optimizations will most propably be done somewhen after the big code cleanup.

    Quote Originally Posted by Bulat Ziganshin
    2) check first the most probable ones - last used, typically used (such as -6/-9 settings) and check in (very) fast mode only these typical settings
    This is already done for the last/most used modes. Precomp uses a priority list to keep track of the modes used. If one of the recompression steps returns complete success (all decompressed bytes could be recompressed), the remaining modes are skipped.

    Using typical settings is also supported - just use "-c69" or whatever suites your needs
    http://schnaader.info
    Damn kids. They're all alike.

  30. #30
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    561
    Thanks
    212
    Thanked 200 Times in 93 Posts
    I just recompiled lprepaq without the ftell "bug", but it is only in the compression part and only speeds up compression about 10%...

    Recompiled version available at http://schnaader.info

    G++ really seems to be faster than my MSVC compile. Even lpaq1 is slower if I compile it myself. Using -Ox or -O2 doesn't make a difference...

    Perhaps someone could try to compile it with another MSVC version or with G++ (not sure if the .lib file works in that case)...

    lpaq1_self_compiled 3 packjpg22.dll packjpg22.lp3
    126976 -> 122160 in 3.96 sec., using 27 MB memory.

    lpaq1 3 packjpg22.dll packjpg22.lp3
    126976 -> 122160 in 2.62 sec., using 27 MB memory.
    http://schnaader.info
    Damn kids. They're all alike.

Page 1 of 2 12 LastLast

Similar Threads

  1. Precomp v0.3.7 and lprepaq v1.3
    By schnaader in forum Forum Archive
    Replies: 40
    Last Post: 11th November 2007, 23:18

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •