Page 1 of 2 12 LastLast
Results 1 to 30 of 41

Thread: Precomp v0.3.7 and lprepaq v1.3

  1. #1
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    571
    Thanks
    219
    Thanked 205 Times in 97 Posts
    Hi!

    I just uploaded Precomp 0.3.7 and lprepaq v1.3. Major changes:

    - Updated packJPG to version 2.3 (10% faster and progressive JPG support).
    - Switched compiler for both Precomp and lprepaq to MinGW g++. lprepaq compression/decompression step is much faster now.

    Have a look at http://schnaader.info
    http://schnaader.info
    Damn kids. They're all alike.

  2. #2
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    408
    Thanks
    0
    Thanked 5 Times in 5 Posts
    Nice release!! I look forward to every new version of precomp. Those programs are a big step for data compression. That is only a fast release, right? When can we hope for multi pass and other file formats? That would be a very good program for many sorts of compressed file formats (images, common game engines...).

  3. #3
    Programmer Jan Ondrus's Avatar
    Join Date
    Sep 2008
    Location
    Rychnov nad Kněžnou, Czech Republic
    Posts
    278
    Thanks
    33
    Thanked 137 Times in 49 Posts
    I tried combining lprepaq source/precomp.dll with another paq version but was not succesfull. "precompress_file" procedure crashed when called more than once.

  4. #4
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    571
    Thanks
    219
    Thanked 205 Times in 97 Posts
    Quote Originally Posted by Jan Ondrus
    I tried combining lprepaq source/precomp.dll with another paq version but was not succesfull. "precompress_file" procedure crashed when called more than once.
    Yes, I can indeed reproduce this with my DLL test program. Ill have a look at why this happens and try to correct it, thanks!

    Quote Originally Posted by Simon Berger
    That is only a fast release, right? When can we hope for multi pass and other file formats? That would be a very good program for many sorts of compressed file formats (images, common game engines...).
    It is a fast release, yes. Multipass is on the top of my todo-list and will hopefully be relatively easy to implement.
    http://schnaader.info
    Damn kids. They're all alike.

  5. #5
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    571
    Thanks
    219
    Thanked 205 Times in 97 Posts
    I fixed the Precomp DLL crash. lprepaq.zip is reuploaded and contains the fixed DLL, but you can also download the DLL directly here:

    http://rapidshare.com/files/66650812/precomp.dll.h tml

    It would be nice if you manage to merge Precomp and PAQ, Jan. I would have tried this next, but I don't have much time at the moment and I don't know the PAQ source code (especially the new one) that well. Good luck!
    http://schnaader.info
    Damn kids. They're all alike.

  6. #6
    Member
    Join Date
    Jul 2008
    Posts
    54
    Thanks
    0
    Thanked 0 Times in 0 Posts
    What about a better zlib detection and zip compression too?, i tried some zlib files and dont' works and with zip the same
    And maybe this is for future...you know LZO?

  7. #7
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    571
    Thanks
    219
    Thanked 205 Times in 97 Posts
    Quote Originally Posted by John
    What about a better zlib detection and zip compression too?, i tried some zlib files and dont works and with zip the same
    Large files inside ZIP can often be decompressed, but not recompressed bit-to-bit identical (you can see that if you use the debug switch -v). This will change with "lossy" modes where the decompressed stream is the same, but the recompressed stream is not bit-to-bit identical.
    About zlib files: Did you try -slow mode? If this also doesnt work, could you e-mail me the files, then Id have a look.

    Quote Originally Posted by John
    And maybe this is for future...you know LZO?
    Ill have a look, but implementing new LZ-algorithms has a very low priority right now.
    http://schnaader.info
    Damn kids. They're all alike.

  8. #8
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,511
    Thanks
    746
    Thanked 668 Times in 361 Posts
    Quote Originally Posted by John
    And maybe this is for future...you know LZO?
    why you need LZO support? i dont see any reasons

  9. #9
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Thanks Christian!

  10. #10
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    571
    Thanks
    219
    Thanked 205 Times in 97 Posts
    Made a quick test and compared with the results LovePimple posted here:

    filename > old result (lprepaq v1.1 option > new result (lprepaq v1.3 option 6) [comments]

    A10.jpg > 699,694 > 698,583
    AcroRd32.exe > 1,335,983 > 1,158,804 [lpaq6 E8E9 filter]
    english.dic > 461,797 > 428,011
    FlashMX.pdf > 2,329,859 > 2,233,647
    FP.LOG > 402,393 > 370,132
    MSO97.DLL > 1,687,334 > 1,568,371 [lpaq6 E8E9 filter]
    ohs.doc > 740,378 > 673,779 [big JPG handled correctly now, progressive JPG support]
    rafale.bmp > 728,270 > 724,485
    vcfiu.hlp > 513,283 > 494,065
    world95.txt > 438,541 > 431,288

    Total > 9,337,532 bytes > 8,781,165 bytes [even better than PAQ8O6 (8,859,59!]

    Has someone got results for option 8 yet?

    Combining PAQ8O6 and Precomp would be very interesting
    http://schnaader.info
    Damn kids. They're all alike.

  11. #11
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,511
    Thanks
    746
    Thanked 668 Times in 361 Posts
    not bad

  12. #12
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    408
    Thanks
    0
    Thanked 5 Times in 5 Posts
    lprepaq.exe -slow 8 A10.jpg
    842468 -> 698585 in 3.454 sec. using 774 MB memory

    lprepaq.exe -slow 8 AcroRd32.exe
    3870784 -> 1165012 in 24.906 sec. using 774 MB memory

    lprepaq.exe -slow 8 english.dic
    4067439 -> 426994 in 6.657 sec. using 774 MB memory

    lprepaq.exe -slow 8 FlashMX.pdf
    4526946 -> 2228848 in 40.547 sec. using 774 MB memory

    lprepaq.exe -slow 8 FP.LOG
    20617071 -> 366225 in 24.984 sec. using 774 MB memory

    lprepaq.exe -slow 8 MSO97.DLL
    3782416 -> 1584742 in 24.250 sec. using 774 MB memory

    lprepaq.exe -slow 8 ohs.doc
    4168192 -> 673616 in 7.078 sec. using 774 MB memory

    lprepaq.exe -slow 8 rafale.bmp
    4149414 -> 724404 in 6.547 sec. using 774 MB memory

    lprepaq.exe -slow 8 vcfiu.hlp
    4121418 -> 494072 in 31.469 sec. using 774 MB memory

    lprepaq.exe -slow 8 world95.txt
    2988578 -> 430940 in 9.500 sec. using 774 MB memory

    Result: 8793438

    A little bit worse!

  13. #13
    Member
    Join Date
    Feb 2009
    Location
    USA
    Posts
    58
    Thanks
    0
    Thanked 0 Times in 0 Posts
    I'm getting an unhandled exception while running precomp. I get it in the same place running it under Windows XP, as in wine on linux. I am precompressing a 697 Meg tar file with a variety of fiel formats inside. It gets to a 520 Meg pcf file and says "Unhandled exception at 0x77c41165 in precomf.exe: 0xC0000005: Access violation writing location 0x00e4c000". I'll try to run it through a debugger and try to get a stack trace.

  14. #14
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Precomp v0.3.7 + CCMx v1.26b (7) for comparison:

    A10.jpg > 698,278
    AcroRd32.exe > 1,186,021
    english.dic > 475,314
    FlashMX.pdf > 2,372,034
    FP.LOG > 437,605
    MSO97.DLL > 1,607,832
    ohs.doc > 685,151
    rafale.bmp > 763,721
    vcfiu.hlp > 528,339
    world95.txt > 474,068

    Total = 9,228,363 bytes

  15. #15
    Member
    Join Date
    Feb 2009
    Location
    USA
    Posts
    58
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Ok, I wasn't able to get a stack trace, but I was able to narrow it down to which file it's crashing on. Ironically it's a Thumbs.db (one of those annoying files produced by windows). Here's a link to it, so you can reproduce it.

    http://www.mytempdir.com/2052180

  16. #16
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    571
    Thanks
    219
    Thanked 205 Times in 97 Posts
    Quote Originally Posted by Hahobas
    Ok, I wasnt able to get a stack trace, but I was able to narrow it down to which file its crashing on. Ironically its a Thumbs.db (one of those annoying files produced by windows). Heres a link to it, so you can reproduce it.

    http://www.mytempdir.com/2052180
    This is a bug in packJPG, I send the JPG streams that cause that crashes to Matthias Stirner, so he can resolve this. Take 3070 bytes from position 8796 from this thumbs.db, for example. The resulting file will cause packJPG 2.3 to crash. A workaround is to use debug mode (-v) and ignore (-i) the streams that lead to the crash. For this file:

    precomp -i8796 -i20188 thumbs.db

    Another would be to switch off JPG recompression completely (-t-j), but that is of less use for collections of files.

    Simon Berger also send me a similar bug report with the installation from the game Enemy Terrority that crashes Precomp/packJPG, so it seems those invalid JPG files do not only appear in thumbs.db files...

    Anyway, thanks for the bug report and for your efforts to get a stack trace.

    Quote Originally Posted by Simon Berger
    lprepaq.exe -slow 8 [...]
    Result: 8793438
    A little bit worse!
    You should not use -slow mode for this data set because there are no other zLib streams as the detected ones in normal mode (I think), and slow mode leads to many false positives that are decompressed into more or less garbage and irritate the following lpaq step. Without slow mode, result should get better.
    This is a result of the bad zLib stream detection, which is a result of the small zLib header. Eventually I will be able to optimize the detection, but at the moment slow mode should only be used if you know that it will help (for example, if you have SWF, SIS, 3DM or other files containing zLib-streams that are only detected with slow mode at the moment.
    http://schnaader.info
    Damn kids. They're all alike.

  17. #17
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    408
    Thanks
    0
    Thanked 5 Times in 5 Posts
    ok, thought that it didn?t get more compression but not that it is worse. Without the -slow switch it?s also only a small amount better.

    A10.jpg > 698.585
    AcroRd32.exe > 1.158.143
    english.dic > 426.994
    FlashMX.pdf > 2.228.307
    FP.LOG > 366.225
    MSO97.DLL > 1.565.758
    ohs.doc > 673.616
    rafale.bmp > 724.404
    vcfiu.hlp > 493.828
    world95.txt > 430.940

    Result: 8.766.800

  18. #18
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    571
    Thanks
    219
    Thanked 205 Times in 97 Posts
    I changed the JPG detection to avoid passing invalid JPGs to packJPG. This prevents crashes on such JPG streams (for example those in the thumbs.db Hahobas mentioned).
    Version numbers are unchanged because nothing else changed.

    Have a look at http://schnaader.info
    http://schnaader.info
    Damn kids. They're all alike.

  19. #19
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Thanks Christian!

  20. #20
    Programmer Jan Ondrus's Avatar
    Join Date
    Sep 2008
    Location
    Rychnov nad Kněžnou, Czech Republic
    Posts
    278
    Thanks
    33
    Thanked 137 Times in 49 Posts
    paq8o6pre - Combined source from lprepaq.cpp file (pre)compressor using precomp.dll (C) 2007 Christian Schneider and paq8o6pre file compressor/archiver. (Release by KZ, Sept. 28, 2007)
    Contains Microsoft Visual C++ project including paq8o6pre.exe.
    - Different behaviour when extracting and file to be extracted already exists. Comparing files not possible due to precompressing step.
    - Test result:
    senszonen.svgz (http://www.ark.in-berlin.de/senszonen.svgz)
    senszonen.svgz - 27711
    senszonen.svgz.paq8o6 - 27554
    senszonen.svgz.paq8o6pre - 15017
    senszonen.svgz.lprepaq13 - 17681

    http://www.mytempdir.com/2054697

  21. #21
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Thanks Jan!

    Mirror: Download

  22. #22
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    571
    Thanks
    219
    Thanked 205 Times in 97 Posts
    Quote Originally Posted by Jan Ondrus
    paq8o6pre - Combined source from lprepaq.cpp file (pre)compressor using precomp.dll (C) 2007 Christian Schneider and paq8o6pre file compressor/archiver. (Release by KZ, Sept. 28, 2007)
    Contains Microsoft Visual C++ project including paq8o6pre.exe.
    Good work!

    Some notes:
    - zlib1.dll should be included
    - Changing Precomp switches is not supported yet. Perhaps Ill try to add this.
    http://schnaader.info
    Damn kids. They're all alike.

  23. #23
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by schnaader
    zlib1.dll should be included
    Added to mirror.

  24. #24
    Programmer Jan Ondrus's Avatar
    Join Date
    Sep 2008
    Location
    Rychnov nad Kněžnou, Czech Republic
    Posts
    278
    Thanks
    33
    Thanked 137 Times in 49 Posts
    paq8o8pre:
    - updated to paq8o8 version
    - zlib1.dll added
    - precomp switches added

    http://www.mytempdir.com/2056220

  25. #25
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Thanks Jan!

    Mirror: Download (338 KB)

  26. #26
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    571
    Thanks
    219
    Thanked 205 Times in 97 Posts
    Quote Originally Posted by Jan Ondrus
    paq8o8pre:
    - updated to paq8o8 version
    - zlib1.dll added
    - precomp switches added

    http://www.mytempdir.com/2056220
    Well done!

    I also put it on my site: http://schnaader.info
    http://schnaader.info
    Damn kids. They're all alike.

  27. #27
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts
    LPREPAQ V.1.3
    New Monster of Compression

  28. #28
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quick test of PAQ8O8pre...

    All files were compressed using the -7 setting.

    A10.jpg > 638,165
    AcroRd32.exe > 910,920
    english.dic > 385,616
    FlashMX.pdf > 1,856,842
    FP.LOG > 264,891
    MSO97.DLL > 1,293,727
    ohs.doc > 483,023
    rafale.bmp > 551,490
    vcfiu.hlp > 401,854
    world95.txt > 358,499

    Total = 7,145,027 bytes


    Not a bad result for the -7 setting!

  29. #29
    Member
    Join Date
    May 2008
    Location
    Earth
    Posts
    115
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Will it also be possible to precompress .hlp and .chm files?

  30. #30
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    408
    Thanks
    0
    Thanked 5 Times in 5 Posts
    I don?t think it is possible to lossless recompress chm files. But yes at least for a lossy mode both formats would be very good to support

Page 1 of 2 12 LastLast

Similar Threads

  1. Precomp 0.4
    By schnaader in forum Data Compression
    Replies: 190
    Last Post: 5th October 2010, 16:13
  2. Precomp (and Precomp Comfort) in 315 kb
    By Yuri Grille. in forum Data Compression
    Replies: 2
    Last Post: 1st April 2009, 20:40
  3. Precomp 0.3.8
    By schnaader in forum Data Compression
    Replies: 116
    Last Post: 6th March 2009, 10:37
  4. LPREPAQ released!
    By LovePimple in forum Forum Archive
    Replies: 36
    Last Post: 5th October 2007, 16:03
  5. Precomp 0.3.3 is out!
    By squxe in forum Forum Archive
    Replies: 1
    Last Post: 20th July 2007, 18:27

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •