paq8lab 1.0 archiver

**bwt** · 21st November 2019, 02:02

this is forked from paq8pxd. i include the source, maybe the result is 15.9xx.xxx bytes for enwik8 using -s14 option.

**Shelwien** · 21st November 2019, 02:49

Comparing to v47_6 the difference is this:

Code:

1c1
< /* paq8pxd file compressor/archiver.  Release by Kaido Orav, Mar. 4, 2018
---
> /* paq8lab file compressor/archiver.  forked from paq8pxd 
552c552
< #define PROGNAME "paq8pxd47_6"  // Please change this if you change the program.
---
> #define PROGNAME "paq8lab 1.0"  // Please change this if you change the program.
5741a5742,5750
> 
>        cm.set(hashxxk(507,lastLetter,x.col,x.frstchar,x.spaces));
>        cm.set(hashxxk(508,x.frstchar,x.spaces,x.col,wordGap));
>        cm.set(hashxxk(509,text0,x.col,x.spaces));     
> 
> 
>        cm.set(hashxxk(510,wrdhsh,lastUpper<x.wordlen,lastLetter,(lastUpper<x.col)));     
>         cm.set(hashxxk(511,wrdhsh,firstLetter,lastLetter,(lastUpper<x.col)));//
>

**bwt** · 21st November 2019, 03:23

Originally Posted by Shelwien

Comparing to v47_6 the difference is this:

Code:

1c1
< /* paq8pxd file compressor/archiver.  Release by Kaido Orav, Mar. 4, 2018
---
> /* paq8lab file compressor/archiver.  forked from paq8pxd 
552c552
< #define PROGNAME "paq8pxd47_6"  // Please change this if you change the program.
---
> #define PROGNAME "paq8lab 1.0"  // Please change this if you change the program.
5741a5742,5750
> 
>        cm.set(hashxxk(507,lastLetter,x.col,x.frstchar,x.spaces));
>        cm.set(hashxxk(508,x.frstchar,x.spaces,x.col,wordGap));
>        cm.set(hashxxk(509,text0,x.col,x.spaces));     
> 
> 
>        cm.set(hashxxk(510,wrdhsh,lastUpper<x.wordlen,lastLetter,(lastUpper<x.col)));     
>         cm.set(hashxxk(511,wrdhsh,firstLetter,lastLetter,(lastUpper<x.col)));//
>

Yes. Please test it for enwik8 using -s14

**Darek** · 21st November 2019, 03:55

@bwt - Could you compile Win64 binary?

And second question: why you didn't use paq8pxd v48_bwt4 as a base?

For paq8pxd v48_bwt4 I've got such scores:

16'429'216 - enwik8 -s6 by Paq8pxd_v48_bwt4 (your score)
16'183'320 - enwik8 -s8 by Paq8pxd_v48_bwt4
15'977'376 - enwik8 -s14 by Paq8pxd_v48_bwt4 - the result was below 16'000'000 bytes.
16'091'113 - enwik8.drt -s14 by Paq8pxd_v48_bwt4

**bwt** · 21st November 2019, 04:27

Originally Posted by Darek

@bwt - Could you compile Win64 binary?

And second question: why you didn't use paq8pxd v48_bwt4 as a base?

For paq8pxd v48_bwt4 I've got such scores:

16'429'216 - enwik8 -s6 by Paq8pxd_v48_bwt4 (your score)
16'183'320 - enwik8 -s8 by Paq8pxd_v48_bwt4
15'977'376 - enwik8 -s14 by Paq8pxd_v48_bwt4 - the result was below 16'000'000 bytes.
16'091'113 - enwik8.drt -s14 by Paq8pxd_v48_bwt4

I still forget where i put g.bat for it and where the source bwt4 so I download it from encode forum.

**bwt** · 21st November 2019, 05:12

Originally Posted by bwt

I still forget where i put g.bat for it and where the source bwt4 so I download it from encode forum.

Maybe someone in this forum can help to build it

**bwt** · 21st November 2019, 14:20

Originally Posted by Darek

@bwt - Could you compile Win64 binary?

And second question: why you didn't use paq8pxd v48_bwt4 as a base?

For paq8pxd v48_bwt4 I've got such scores:

16'429'216 - enwik8 -s6 by Paq8pxd_v48_bwt4 (your score)
16'183'320 - enwik8 -s8 by Paq8pxd_v48_bwt4
15'977'376 - enwik8 -s14 by Paq8pxd_v48_bwt4 - the result was below 16'000'000 bytes.
16'091'113 - enwik8.drt -s14 by Paq8pxd_v48_bwt4

How about enwik9 ? Have you tested it ?

**Darek** · 21st November 2019, 21:04

I've tried ywo times. Unfortunatelly both attempts ends without success. This version uses more than 32GB of memory and (besides much more timeconsuming) crashes at about 80% of the file - I don't know if it was in the same moment...

My last scores are:
16'525'916 - enwik8 -s6 by Paq8pxd_v48_bwt1 - your score
16'248'408 - enwik8 -s8 by Paq8pxd_v48_bwt1
16'004'759 - enwik8 -s14 by Paq8pxd_v48_bwt1
16'088'990 - enwik8.drt -s14 by Paq8pxd_v48_bwt1
136'732'561 - enwik9 -s6 by Paq8pxd_v48_bwt1 - your score
126'183'029 - enwik9_1423 -s14 by Paq8pxd_v48_bwt1 - the paq series record, submitted to LTCB as 4'th score at now! With -s15 option program crashes...

16'472'556 - enwik8 -s6 by Paq8pxd_v48_bwt2
16'228'952 - enwik8 -s8 by Paq8pxd_v48_bwt2
16'001'495 - enwik8 -s14 by Paq8pxd_v48_bwt2
16'088'506 - enwik8.drt -s14 by Paq8pxd_v48_bwt2
126'157'295 - enwik9_1423 -s14 by Paq8pxd_v48_bwt2 - estimate, program didn't finish even on -s14

16'429'216 - enwik8 -s6 by Paq8pxd_v48_bwt4
16'183'320 - enwik8 -s8 by Paq8pxd_v48_bwt4
15'977'376 - enwik8 -s14 by Paq8pxd_v48_bwt4
16'091'113 - enwik8.drt -s14 by Paq8pxd_v48_bwt4
125'967'139 - enwik9_1423 -s14 by Paq8pxd_v48_bwt4 - very goooood result but as above it's estimate, program didn't finish even on -s14

**moisesmcardona** · 22nd November 2019, 22:37

Originally Posted by Darek

I've tried ywo times. Unfortunatelly both attempts ends without success. This version uses more than 32GB of memory and (besides much more timeconsuming) crashes at about 80% of the file - I don't know if it was in the same moment...

My last scores are:
16'525'916 - enwik8 -s6 by Paq8pxd_v48_bwt1 - your score
16'248'408 - enwik8 -s8 by Paq8pxd_v48_bwt1
16'004'759 - enwik8 -s14 by Paq8pxd_v48_bwt1
16'088'990 - enwik8.drt -s14 by Paq8pxd_v48_bwt1
136'732'561 - enwik9 -s6 by Paq8pxd_v48_bwt1 - your score
126'183'029 - enwik9_1423 -s14 by Paq8pxd_v48_bwt1 - the paq series record, submitted to LTCB as 4'th score at now! With -s15 option program crashes...

16'472'556 - enwik8 -s6 by Paq8pxd_v48_bwt2
16'228'952 - enwik8 -s8 by Paq8pxd_v48_bwt2
16'001'495 - enwik8 -s14 by Paq8pxd_v48_bwt2
16'088'506 - enwik8.drt -s14 by Paq8pxd_v48_bwt2
126'157'295 - enwik9_1423 -s14 by Paq8pxd_v48_bwt2 - estimate, program didn't finish even on -s14

16'429'216 - enwik8 -s6 by Paq8pxd_v48_bwt4
16'183'320 - enwik8 -s8 by Paq8pxd_v48_bwt4
15'977'376 - enwik8 -s14 by Paq8pxd_v48_bwt4
16'091'113 - enwik8.drt -s14 by Paq8pxd_v48_bwt4
125'967'139 - enwik9_1423 -s14 by Paq8pxd_v48_bwt4 - very goooood result but as above it's estimate, program didn't finish even on -s14

Did you managed to compile it? If so, how? I'm trying to but it fails with undeclared stuff.

**Shelwien** · 22nd November 2019, 23:01

It compiles with parts other than .cpp taken from this: https://github.com/kaitz/paq8pxd/releases/tag/v47

**Darek** · 23rd November 2019, 01:21

Originally Posted by moisesmcardona

Did you managed to compile it? If so, how? I'm trying to but it fails with undeclared stuff.

No, there were binaries published on paq8pxd forum.
Like mentioned above v47_6 - https://encode.su/threads/1464-Paq8p...ll=1#post57094

**bwt** · 23rd November 2019, 18:53

Originally Posted by Darek

No, there were binaries published on paq8pxd forum.
Like mentioned above v47_6 - https://encode.su/threads/1464-Paq8p...ll=1#post57094

i have installed mingw 16.1 distro from nuwen.net but when i compiled paq8lab it does not work. Here is the source code and please shelwien or someone help to compile it and run how many enwik8 and enwik9 can be compressed again. thank you

**bwt** · 23rd November 2019, 18:55

this is based on paq8pxd48_bwt1

**Shelwien** · 23rd November 2019, 19:03

Might be too new... try getting gcc7 on mingw-w64.org

**bwt** · 24th November 2019, 10:13

Originally Posted by Shelwien

Might be too new... try getting gcc7 on mingw-w64.org

enwik8
paq8lab_2 -s6 16526213 bytes

**bwt** · 25th November 2019, 05:16

Originally Posted by Shelwien

Might be too new... try getting gcc7 on mingw-w64.org

i have try compile paq8lab_2 using g32.bat from paq8px_v77b but it fail.
if i compile using g32.bat for paq8px_v77b it success. why ?

**Shelwien** · 25th November 2019, 05:37

I used this to compile it

**bwt** · 25th November 2019, 06:26

Originally Posted by Shelwien

I used this to compile it

still can not compile

**Shelwien** · 25th November 2019, 06:52

1. Get this: https://vorboss.dl.sourceforge.net/p...-rt_v5-rev1.7z
2. Install to C:\mingw720x
3. run g.bat in archive above

**bwt** · 25th November 2019, 09:04

Originally Posted by Shelwien

1. Get this: https://vorboss.dl.sourceforge.net/p...-rt_v5-rev1.7z
2. Install to C:\mingw720x
3. run g.bat in archive above

still does not work. may you use devc++ to compile it ?

**bwt** · 25th November 2019, 09:05

i use devc++ 5.11

**bwt** · 25th November 2019, 13:08

Originally Posted by bwt

i use devc++ 5.11

how to include zlib in devc++ 5.11 ??

**bwt** · 28th November 2019, 02:47

Originally Posted by bwt

enwik8
paq8lab_2 -s6 16526213 bytes

enwik9 -s6 option
130731195 bytes

**Darek** · 29th November 2019, 13:16

I've tried to compress with -s14 and -s15 options and there are "out of memory" error. This version even not tried to use swap file...
-s14 option uses, as it wrote, 42GB of memory and (strange) -s15 try to use 39GB of memory.
Instead of this I've testes enwik8 with -s6, -s8 and -s13. Comparison to latest version is like follows:

16'553'942 - enwik8 -s6 by Paq8pxd_v47_5
16'267'313 - enwik8 -s8 by Paq8pxd_v47_5
16'027'128 - enwik8 -s15 by Paq8pxd_v47_5
16'099'328 - enwik8.drt -s15 by Paq8pxd_v47_5
126'387'497 - enwik9_1423 -s15 by Paq8pxd_v47_5 - estimated, program finish to compress but didn't add the header..

16'550'555 - enwik8 -s6 by Paq8pxd_v47_6
16'264'217 - enwik8 -s8 by Paq8pxd_v47_6

16'525'916 - enwik8 -s6 by Paq8pxd_v48_bwt1
16'248'408 - enwik8 -s8 by Paq8pxd_v48_bwt1
16'004'759 - enwik8 -s14 by Paq8pxd_v48_bwt1
16'088'990 - enwik8.drt -s14 by Paq8pxd_v48_bwt1
136'732'561 - enwik9 -s6 by Paq8pxd_v48_bwt1
126'183'029 - enwik9_1423 -s14 by Paq8pxd_v48_bwt1

16'472'556 - enwik8 -s6 by Paq8pxd_v48_bwt2
16'228'952 - enwik8 -s8 by Paq8pxd_v48_bwt2
16'001'495 - enwik8 -s14 by Paq8pxd_v48_bwt2
16'088'506 - enwik8.drt -s14 by Paq8pxd_v48_bwt2
126'157'295 - enwik9_1423 -s14 by Paq8pxd_v48_bwt2 - estimated due to crashes

16'429'216 - enwik8 -s6 by Paq8pxd_v48_bwt4
16'183'320 - enwik8 -s8 by Paq8pxd_v48_bwt4
15'977'376 - enwik8 -s14 by Paq8pxd_v48_bwt4
16'091'113 - enwik8.drt -s14 by Paq8pxd_v48_bwt4
125'967'139 - enwik9_1423 -s14 by Paq8pxd_v48_bwt4 - estimated due to issues

16'526'213 - enwik8 -s6 by Paq8lab_2
16'249'207 - enwik8 -s8 by Paq8lab_2
16'024'588 - enwik8 -s13 by Paq8lab_2

That looks that paq8lab_2 version is very similar to bwt1 (slightly worse), then my estimate for -s14/15 is as below:
16'005'546 - enwik8 -s14 by Paq8lab_2 - estimate - unfortunatelly slightly higher than 15'9xx'xxx

16'108'923 - enwik8.drt -s14 by Paq8lab_2 - estimate
126'339'363 - enwik9_1423 -s14 by Paq8lab_2 - estimate

**bwt** · 12th January 2020, 02:52

Originally Posted by Shelwien

I used this to compile it

this is happen while using g.bat

**Shelwien** · 12th January 2020, 03:51

There's a commented out line above that:

Code:

    //U64 scompsize=0;

Also it'd compile if you didn't remove -DNDEBUG:

Code:

                            #ifndef NDEBUG 
                            printf("Stream(%d) block from %0lu to %0lu bytes\n",i,datasegmentlen, out->curpos()-scompsize);
                            scompsize= out->curpos();
                            #endif

**bwt** · 12th January 2020, 04:12

Originally Posted by Shelwien

There's a commented out line above that:

Code:

    //U64 scompsize=0;

Also it'd compile if you didn't remove -DNDEBUG:

Code:

                            #ifndef NDEBUG 
                            printf("Stream(%d) block from %0lu to %0lu bytes\n",i,datasegmentlen, out->curpos()-scompsize);
                            scompsize= out->curpos();
                            #endif

still happen error when i have removed -DNDEBUG in g.bat

**Shelwien** · 12th January 2020, 04:36

Change this:

Code:

//U64 scompsize=0;

to this:

Code:

U64 scompsize=0;

**bwt** · 12th January 2020, 06:03

Originally Posted by Shelwien

Change this:

Code:

//U64 scompsize=0;

to this:

Code:

U64 scompsize=0;

**bwt** · 23rd February 2020, 11:56

paq8pxdv48_bwt2
@shelwien could you compile this source code please ? maybe could you remove zlib n gif function so i can compile it my self please ? thank you

Thread: paq8lab 1.0 archiver

Thread Tools

Search Thread

Display

paq8lab 1.0 archiver

Thanks:

Thanks:

Thanks:

Thanks (2):

Thanks:

Thanks:

Similar Threads

B1 Archiver

CSA archiver

paf archiver

Reasonable Archiver

PeaZip Archiver

Posting Permissions