Page 1 of 2 12 LastLast
Results 1 to 30 of 36

Thread: paq8lab 1.0 archiver

  1. #1
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    84
    Thanks
    15
    Thanked 5 Times in 5 Posts

    paq8lab 1.0 archiver

    this is forked from paq8pxd. i include the source, maybe the result is 15.9xx.xxx bytes for enwik8 using -s14 option.
    Attached Files Attached Files

  2. #2
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,977
    Thanks
    296
    Thanked 1,304 Times in 740 Posts
    Comparing to v47_6 the difference is this:
    Code:
    1c1
    < /* paq8pxd file compressor/archiver.  Release by Kaido Orav, Mar. 4, 2018
    ---
    > /* paq8lab file compressor/archiver.  forked from paq8pxd 
    552c552
    < #define PROGNAME "paq8pxd47_6"  // Please change this if you change the program.
    ---
    > #define PROGNAME "paq8lab 1.0"  // Please change this if you change the program.
    5741a5742,5750
    > 
    >        cm.set(hashxxk(507,lastLetter,x.col,x.frstchar,x.spaces));
    >        cm.set(hashxxk(508,x.frstchar,x.spaces,x.col,wordGap));
    >        cm.set(hashxxk(509,text0,x.col,x.spaces));     
    > 
    > 
    >        cm.set(hashxxk(510,wrdhsh,lastUpper<x.wordlen,lastLetter,(lastUpper<x.col)));     
    >         cm.set(hashxxk(511,wrdhsh,firstLetter,lastLetter,(lastUpper<x.col)));//
    >

  3. Thanks:

    Mike (21st November 2019)

  4. #3
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    84
    Thanks
    15
    Thanked 5 Times in 5 Posts
    Quote Originally Posted by Shelwien View Post
    Comparing to v47_6 the difference is this:
    Code:
    1c1
    < /* paq8pxd file compressor/archiver.  Release by Kaido Orav, Mar. 4, 2018
    ---
    > /* paq8lab file compressor/archiver.  forked from paq8pxd 
    552c552
    < #define PROGNAME "paq8pxd47_6"  // Please change this if you change the program.
    ---
    > #define PROGNAME "paq8lab 1.0"  // Please change this if you change the program.
    5741a5742,5750
    > 
    >        cm.set(hashxxk(507,lastLetter,x.col,x.frstchar,x.spaces));
    >        cm.set(hashxxk(508,x.frstchar,x.spaces,x.col,wordGap));
    >        cm.set(hashxxk(509,text0,x.col,x.spaces));     
    > 
    > 
    >        cm.set(hashxxk(510,wrdhsh,lastUpper<x.wordlen,lastLetter,(lastUpper<x.col)));     
    >         cm.set(hashxxk(511,wrdhsh,firstLetter,lastLetter,(lastUpper<x.col)));//
    >
    Yes. Please test it for enwik8 using -s14

  5. #4
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,217
    Thanks
    743
    Thanked 495 Times in 383 Posts
    @bwt - Could you compile Win64 binary?

    And second question: why you didn't use paq8pxd v48_bwt4 as a base?

    For paq8pxd v48_bwt4 I've got such scores:

    16'429'216 - enwik8 -s6 by Paq8pxd_v48_bwt4 (your score)
    16'183'320 - enwik8 -s8 by Paq8pxd_v48_bwt4
    15'977'376 - enwik8 -s14 by Paq8pxd_v48_bwt4 - the result was below 16'000'000 bytes.
    16'091'113 - enwik8.drt -s14 by Paq8pxd_v48_bwt4

  6. #5
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    84
    Thanks
    15
    Thanked 5 Times in 5 Posts
    Quote Originally Posted by Darek View Post
    @bwt - Could you compile Win64 binary?

    And second question: why you didn't use paq8pxd v48_bwt4 as a base?

    For paq8pxd v48_bwt4 I've got such scores:

    16'429'216 - enwik8 -s6 by Paq8pxd_v48_bwt4 (your score)
    16'183'320 - enwik8 -s8 by Paq8pxd_v48_bwt4
    15'977'376 - enwik8 -s14 by Paq8pxd_v48_bwt4 - the result was below 16'000'000 bytes.
    16'091'113 - enwik8.drt -s14 by Paq8pxd_v48_bwt4
    I still forget where i put g.bat for it and where the source bwt4 so I download it from encode forum.

  7. #6
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    84
    Thanks
    15
    Thanked 5 Times in 5 Posts
    Quote Originally Posted by bwt View Post
    I still forget where i put g.bat for it and where the source bwt4 so I download it from encode forum.
    Maybe someone in this forum can help to build it

  8. #7
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    84
    Thanks
    15
    Thanked 5 Times in 5 Posts
    Quote Originally Posted by Darek View Post
    @bwt - Could you compile Win64 binary?

    And second question: why you didn't use paq8pxd v48_bwt4 as a base?

    For paq8pxd v48_bwt4 I've got such scores:

    16'429'216 - enwik8 -s6 by Paq8pxd_v48_bwt4 (your score)
    16'183'320 - enwik8 -s8 by Paq8pxd_v48_bwt4
    15'977'376 - enwik8 -s14 by Paq8pxd_v48_bwt4 - the result was below 16'000'000 bytes.
    16'091'113 - enwik8.drt -s14 by Paq8pxd_v48_bwt4
    How about enwik9 ? Have you tested it ?

  9. #8
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,217
    Thanks
    743
    Thanked 495 Times in 383 Posts
    I've tried ywo times. Unfortunatelly both attempts ends without success. This version uses more than 32GB of memory and (besides much more timeconsuming) crashes at about 80% of the file - I don't know if it was in the same moment...

    My last scores are:
    16'525'916 - enwik8 -s6 by Paq8pxd_v48_bwt1 - your score
    16'248'408 - enwik8 -s8 by Paq8pxd_v48_bwt1
    16'004'759 - enwik8 -s14 by Paq8pxd_v48_bwt1
    16'088'990 - enwik8.drt -s14 by Paq8pxd_v48_bwt1
    136'732'561 - enwik9 -s6 by Paq8pxd_v48_bwt1 - your score
    126'183'029 - enwik9_1423 -s14 by Paq8pxd_v48_bwt1 - the paq series record, submitted to LTCB as 4'th score at now! With -s15 option program crashes...


    16'472'556 - enwik8 -s6 by Paq8pxd_v48_bwt2
    16'228'952 - enwik8 -s8 by Paq8pxd_v48_bwt2
    16'001'495 - enwik8 -s14 by Paq8pxd_v48_bwt2
    16'088'506 - enwik8.drt -s14 by Paq8pxd_v48_bwt2
    126'157'295 - enwik9_1423 -s14 by Paq8pxd_v48_bwt2 - estimate, program didn't finish even on -s14


    16'429'216 - enwik8 -s6 by Paq8pxd_v48_bwt4
    16'183'320 - enwik8 -s8 by Paq8pxd_v48_bwt4
    15'977'376 - enwik8 -s14 by Paq8pxd_v48_bwt4
    16'091'113 - enwik8.drt -s14 by Paq8pxd_v48_bwt4
    125'967'139 - enwik9_1423 -s14 by Paq8pxd_v48_bwt4 - very goooood result but as above it's estimate, program didn't finish even on -s14

  10. #9
    Member
    Join Date
    Jun 2009
    Location
    Puerto Rico
    Posts
    251
    Thanks
    138
    Thanked 52 Times in 39 Posts
    Quote Originally Posted by Darek View Post
    I've tried ywo times. Unfortunatelly both attempts ends without success. This version uses more than 32GB of memory and (besides much more timeconsuming) crashes at about 80% of the file - I don't know if it was in the same moment...

    My last scores are:
    16'525'916 - enwik8 -s6 by Paq8pxd_v48_bwt1 - your score
    16'248'408 - enwik8 -s8 by Paq8pxd_v48_bwt1
    16'004'759 - enwik8 -s14 by Paq8pxd_v48_bwt1
    16'088'990 - enwik8.drt -s14 by Paq8pxd_v48_bwt1
    136'732'561 - enwik9 -s6 by Paq8pxd_v48_bwt1 - your score
    126'183'029 - enwik9_1423 -s14 by Paq8pxd_v48_bwt1 - the paq series record, submitted to LTCB as 4'th score at now! With -s15 option program crashes...


    16'472'556 - enwik8 -s6 by Paq8pxd_v48_bwt2
    16'228'952 - enwik8 -s8 by Paq8pxd_v48_bwt2
    16'001'495 - enwik8 -s14 by Paq8pxd_v48_bwt2
    16'088'506 - enwik8.drt -s14 by Paq8pxd_v48_bwt2
    126'157'295 - enwik9_1423 -s14 by Paq8pxd_v48_bwt2 - estimate, program didn't finish even on -s14


    16'429'216 - enwik8 -s6 by Paq8pxd_v48_bwt4
    16'183'320 - enwik8 -s8 by Paq8pxd_v48_bwt4
    15'977'376 - enwik8 -s14 by Paq8pxd_v48_bwt4
    16'091'113 - enwik8.drt -s14 by Paq8pxd_v48_bwt4
    125'967'139 - enwik9_1423 -s14 by Paq8pxd_v48_bwt4 - very goooood result but as above it's estimate, program didn't finish even on -s14
    Did you managed to compile it? If so, how? I'm trying to but it fails with undeclared stuff.

  11. #10
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,977
    Thanks
    296
    Thanked 1,304 Times in 740 Posts
    It compiles with parts other than .cpp taken from this: https://github.com/kaitz/paq8pxd/releases/tag/v47

  12. Thanks:

    Darek (23rd November 2019)

  13. #11
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,217
    Thanks
    743
    Thanked 495 Times in 383 Posts
    Quote Originally Posted by moisesmcardona View Post
    Did you managed to compile it? If so, how? I'm trying to but it fails with undeclared stuff.
    No, there were binaries published on paq8pxd forum.
    Like mentioned above v47_6 - https://encode.su/threads/1464-Paq8p...ll=1#post57094

  14. #12
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    84
    Thanks
    15
    Thanked 5 Times in 5 Posts
    Quote Originally Posted by Darek View Post
    No, there were binaries published on paq8pxd forum.
    Like mentioned above v47_6 - https://encode.su/threads/1464-Paq8p...ll=1#post57094

    i have installed mingw 16.1 distro from nuwen.net but when i compiled paq8lab it does not work. Here is the source code and please shelwien or someone help to compile it and run how many enwik8 and enwik9 can be compressed again. thank you
    Attached Files Attached Files

  15. Thanks:

    Darek (23rd November 2019)

  16. #13
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    84
    Thanks
    15
    Thanked 5 Times in 5 Posts
    this is based on paq8pxd48_bwt1

  17. #14
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,977
    Thanks
    296
    Thanked 1,304 Times in 740 Posts
    Might be too new... try getting gcc7 on mingw-w64.org
    Attached Files Attached Files

  18. Thanks (2):

    bwt (24th November 2019),Darek (23rd November 2019)

  19. #15
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    84
    Thanks
    15
    Thanked 5 Times in 5 Posts
    Quote Originally Posted by Shelwien View Post
    Might be too new... try getting gcc7 on mingw-w64.org

    enwik8
    paq8lab_2 -s6 16526213 bytes

  20. #16
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    84
    Thanks
    15
    Thanked 5 Times in 5 Posts
    Quote Originally Posted by Shelwien View Post
    Might be too new... try getting gcc7 on mingw-w64.org
    i have try compile paq8lab_2 using g32.bat from paq8px_v77b but it fail.
    if i compile using g32.bat for paq8px_v77b it success. why ?

  21. #17
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,977
    Thanks
    296
    Thanked 1,304 Times in 740 Posts
    I used this to compile it
    Attached Files Attached Files

  22. Thanks:

    bwt (25th November 2019)

  23. #18
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    84
    Thanks
    15
    Thanked 5 Times in 5 Posts
    Quote Originally Posted by Shelwien View Post
    I used this to compile it

    ​still can not compile

  24. #19
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,977
    Thanks
    296
    Thanked 1,304 Times in 740 Posts
    1. Get this: https://vorboss.dl.sourceforge.net/p...-rt_v5-rev1.7z
    2. Install to C:\mingw720x
    3. run g.bat in archive above

  25. #20
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    84
    Thanks
    15
    Thanked 5 Times in 5 Posts
    Quote Originally Posted by Shelwien View Post
    1. Get this: https://vorboss.dl.sourceforge.net/p...-rt_v5-rev1.7z
    2. Install to C:\mingw720x
    3. run g.bat in archive above

    still does not work. may you use devc++ to compile it ?

  26. #21
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    84
    Thanks
    15
    Thanked 5 Times in 5 Posts
    i use devc++ 5.11

  27. #22
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    84
    Thanks
    15
    Thanked 5 Times in 5 Posts
    Quote Originally Posted by bwt View Post
    i use devc++ 5.11
    how to include zlib in devc++ 5.11 ??

  28. #23
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    84
    Thanks
    15
    Thanked 5 Times in 5 Posts
    Quote Originally Posted by bwt View Post
    enwik8
    paq8lab_2 -s6 16526213 bytes

    enwik9 -s6 option
    130731195 bytes

  29. #24
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,217
    Thanks
    743
    Thanked 495 Times in 383 Posts
    I've tried to compress with -s14 and -s15 options and there are "out of memory" error. This version even not tried to use swap file...
    -s14 option uses, as it wrote, 42GB of memory and (strange) -s15 try to use 39GB of memory.
    Instead of this I've testes enwik8 with -s6, -s8 and -s13. Comparison to latest version is like follows:

    16'553'942 - enwik8 -s6 by Paq8pxd_v47_5
    16'267'313 - enwik8 -s8 by Paq8pxd_v47_5
    16'027'128 - enwik8 -s15 by Paq8pxd_v47_5
    16'099'328 - enwik8.drt -s15 by Paq8pxd_v47_5
    126'387'497 - enwik9_1423 -s15 by Paq8pxd_v47_5 - estimated, program finish to compress but didn't add the header..

    16'550'555 - enwik8 -s6 by Paq8pxd_v47_6
    16'264'217 - enwik8 -s8 by Paq8pxd_v47_6

    16'525'916 - enwik8 -s6 by Paq8pxd_v48_bwt1
    16'248'408 - enwik8 -s8 by Paq8pxd_v48_bwt1
    16'004'759 - enwik8 -s14 by Paq8pxd_v48_bwt1
    16'088'990 - enwik8.drt -s14 by Paq8pxd_v48_bwt1
    136'732'561 - enwik9 -s6 by Paq8pxd_v48_bwt1
    126'183'029 - enwik9_1423 -s14 by Paq8pxd_v48_bwt1


    16'472'556 - enwik8 -s6 by Paq8pxd_v48_bwt2
    16'228'952 - enwik8 -s8 by Paq8pxd_v48_bwt2
    16'001'495 - enwik8 -s14 by Paq8pxd_v48_bwt2
    16'088'506 - enwik8.drt -s14 by Paq8pxd_v48_bwt2
    126'157'295 - enwik9_1423 -s14 by Paq8pxd_v48_bwt2 - estimated due to crashes

    16'429'216 - enwik8 -s6 by Paq8pxd_v48_bwt4
    16'183'320 - enwik8 -s8 by Paq8pxd_v48_bwt4
    15'977'376 - enwik8 -s14 by Paq8pxd_v48_bwt4
    16'091'113 - enwik8.drt -s14 by Paq8pxd_v48_bwt4
    125'967'139 - enwik9_1423 -s14 by Paq8pxd_v48_bwt4 - estimated due to issues

    16'526'213 - enwik8 -s6 by Paq8lab_2
    16'249'207 - enwik8 -s8 by Paq8lab_2
    16'024'588 - enwik8 -s13 by Paq8lab_2

    That looks that paq8lab_2 version is very similar to bwt1 (slightly worse), then my estimate for -s14/15 is as below:
    16'005'546 - enwik8 -s14 by Paq8lab_2 - estimate - unfortunatelly slightly higher than 15'9xx'xxx
    16'108'923 - enwik8.drt -s14 by Paq8lab_2 - estimate
    126'339'363 - enwik9_1423 -s14 by Paq8lab_2 - estimate

  30. Thanks:

    bwt (29th November 2019)

  31. #25
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    84
    Thanks
    15
    Thanked 5 Times in 5 Posts
    Quote Originally Posted by Shelwien View Post
    I used this to compile it
    this is happen while using g.bat
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	paq8lapcompile.PNG 
Views:	36 
Size:	35.5 KB 
ID:	7237  

  32. #26
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,977
    Thanks
    296
    Thanked 1,304 Times in 740 Posts
    There's a commented out line above that:
    Code:
        //U64 scompsize=0;
    Also it'd compile if you didn't remove -DNDEBUG:
    Code:
                                #ifndef NDEBUG 
                                printf("Stream(%d) block from %0lu to %0lu bytes\n",i,datasegmentlen, out->curpos()-scompsize);
                                scompsize= out->curpos();
                                #endif

  33. #27
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    84
    Thanks
    15
    Thanked 5 Times in 5 Posts
    Quote Originally Posted by Shelwien View Post
    There's a commented out line above that:
    Code:
        //U64 scompsize=0;
    Also it'd compile if you didn't remove -DNDEBUG:
    Code:
                                #ifndef NDEBUG 
                                printf("Stream(%d) block from %0lu to %0lu bytes\n",i,datasegmentlen, out->curpos()-scompsize);
                                scompsize= out->curpos();
                                #endif

    ​still happen error when i have removed -DNDEBUG in g.bat

  34. #28
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,977
    Thanks
    296
    Thanked 1,304 Times in 740 Posts
    Change this:
    Code:
    //U64 scompsize=0;
    to this:
    Code:
    U64 scompsize=0;

  35. #29
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    84
    Thanks
    15
    Thanked 5 Times in 5 Posts
    Quote Originally Posted by Shelwien View Post
    Change this:
    Code:
    //U64 scompsize=0;
    to this:
    Code:
    U64 scompsize=0;
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	paq8laberror.PNG 
Views:	49 
Size:	35.5 KB 
ID:	7239  

  36. #30
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    84
    Thanks
    15
    Thanked 5 Times in 5 Posts
    paq8pxdv48_bwt2
    @shelwien could you compile this source code please ? maybe could you remove zlib n gif function so i can compile it my self please ? thank you
    Attached Files Attached Files

Page 1 of 2 12 LastLast

Similar Threads

  1. B1 Archiver
    By Gonzalo in forum Data Compression
    Replies: 0
    Last Post: 24th November 2014, 21:20
  2. CSA archiver
    By Surfer in forum Data Compression
    Replies: 4
    Last Post: 7th June 2011, 11:18
  3. paf archiver
    By Shelwien in forum Data Compression
    Replies: 25
    Last Post: 5th August 2010, 01:03
  4. Reasonable Archiver
    By LovePimple in forum Forum Archive
    Replies: 7
    Last Post: 12th March 2008, 15:05
  5. PeaZip Archiver
    By LovePimple in forum Forum Archive
    Replies: 8
    Last Post: 22nd August 2007, 15:05

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •