Page 22 of 26 FirstFirst ... 122021222324 ... LastLast
Results 631 to 660 of 762

Thread: Paq8pxd dict

  1. #631
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    465
    Thanks
    193
    Thanked 289 Times in 157 Posts
    Quote Originally Posted by CompressMaster View Post
    Specs?

    Mine:
    Windows 7 Home Premium 64 bit
    Intel Core i3 2310M
    NVIDIA GeForce GT 520M
    Yours has AVX, binary is AVX2 compile.

    Edit:
    In this post https://encode.su/threads/3064-paq8p...ll=1#post60419
    I tested different architectures speed. Not the best file to test.
    KZo


  2. Thanks:

    CompressMaster (7th August 2019)

  3. #632
    Member
    Join Date
    Jun 2018
    Location
    Slovakia
    Posts
    183
    Thanks
    49
    Thanked 13 Times in 13 Posts
    @kaitz,

    Thanks a lot! Now I´m able to execute it, but...
    When I´ve tried to compress JPEG by drag-and-drop, I´ve got an error "error opening conf.pxv file" (attached)
    File is there and it seems that it isn´t corrupted. I´ve tried to make a shortcut and execute it with admin credentials, but same error.

    Edit: That build contains models that I´m able to alter directly! Many thanks!

    btw, what does it mean NONE? It´s universal regardless CPU architecture?
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	paq8pxv_v12_NONE error opening conf.pxv file.png 
Views:	46 
Size:	9.4 KB 
ID:	6776  
    Last edited by CompressMaster; 7th August 2019 at 20:01. Reason: see "Edit"

  4. #633
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    465
    Thanks
    193
    Thanked 289 Times in 157 Posts
    This is different program, it needs some files from archive.
    JPEG compression on that version is not that good.

    Attached pxd_v66 SSE4.2 compile.
    Attached Files Attached Files
    KZo


  5. #634
    Member
    Join Date
    Jun 2018
    Location
    Slovakia
    Posts
    183
    Thanks
    49
    Thanked 13 Times in 13 Posts
    Quote Originally Posted by brispuss View Post
    Thanks for the offer of compressing the test JPG file using cmix. But even if cmix compresses the JPG file better than other compressors, I can't use cmix to compress JPG files due to lack of computer memory (only 8GB instead of recommended 32GB+). And I can't afford to upgrade my computer for some time yet, unfortunately. So I must respectfully decline the offer at this time.
    Well, if you REALLY want to execute cmix with that small amount of memory, it´s definitely possible, but compression will be terribly slow (approx 10 times).
    Enable swap (page file) via Advanced Computer Option in System settings. Custom settings and specify amount of drive space that will acts as a additional memory.

  6. #635
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    465
    Thanks
    193
    Thanked 289 Times in 157 Posts
    paq8pxd_v67
    • revert ContextMap mix
    • change wordModel
    • in bypass update models
    • change distanceModel, indirectModel, exeModel
    • in default predictor resume bypass after 0x2000 bytes if predict miss found
      instead default bypass match length of 0xfff bytes
    • compressed data stream, still compressed but faster (eol data for now)
    • memory usage tweak to the same level as v64
    Compression is mostly same as v60 was. On some files its better then previous versions.

    Silesia is about 3024**** bytes,
    Maximumcompression set is about 611**** bytes,
    enwik8 is about 1637**** bytes,
    with option -s8
    Attached Files Attached Files
    KZo


  7. Thanks (3):

    Darek (9th August 2019),Mike (9th August 2019),moisesmcardona (9th August 2019)

  8. #636
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,019
    Thanks
    615
    Thanked 416 Times in 314 Posts
    Scores of my testset for paq8pxd v67 - in general - much better than v66 version and as kaitz wrote - similar to v64 version - some files got gains (Q.WK3, M.DBF), some got loses.
    In total = 9KB worse score = -0.09%.
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	paq8pxd_v67.jpg 
Views:	52 
Size:	640.5 KB 
ID:	6787  

  9. Thanks:

    kaitz (9th August 2019)

  10. #637
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,019
    Thanks
    615
    Thanked 416 Times in 314 Posts
    Scores of enwik8/9 compressed by paq8pxd v67 - slightly worse than previous tested version, however this release (AVX2) is about 50% faster than v63!

    16'309'012 - enwik8 -s8 by Paq8pxd_v63, time 9'240s
    15'967'201 - enwik8 -s15 by Paq8pxd_v63, time 8'880s
    16'63'7302 - enwik8.drt -s15 by Paq8pxd_v63, time 11'837s
    126'597'584 - enwik9_1423 -s15 by Paq8pxd_v63, time 98'387s


    16'374'223 - enwik8 -s8 by Paq8pxd_v67_AVX2, +0.40% to v63score, time 6'431s,
    16'048'070 - enwik8 -s15 by Paq8pxd_v67_AVX2, +0.51% to v63 score, time 6'643s
    16'774'998 - enwik8.drt -s15 by Paq8pxd_v67_AVX2, +0.83% to v63 score, time 8'413s
    127'063'602 - enwik9_1423 -s15 by Paq8pxd_v67_AVX2, +0.37% to v63 score, time 66'041s

  11. Thanks:

    kaitz (15th August 2019)

  12. #638
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    465
    Thanks
    193
    Thanked 289 Times in 157 Posts
    paq8pxd_v68
    Code:
    • update im8Mode,im24model,exemodel from paq8px_179
    • compressed stream /MZIP,EOL,ZLIB fail/
    • linearPredictionModel from paq8px_179
    • add charModel into wordModel paq8px_179
    If i add ppm model to v68 and lstm model (lstm_(25, 3, 20, 0.05)) then enwik is about 16243xxx bytes with option -8
    Attached Files Attached Files
    KZo


  13. Thanks (3):

    Darek (16th August 2019),Mike (15th August 2019),moisesmcardona (15th August 2019)

  14. #639
    Member
    Join Date
    Sep 2015
    Location
    Italy
    Posts
    268
    Thanks
    111
    Thanked 153 Times in 112 Posts
    > lstm_(25, 3, 20, 0.05)
    What implementation do you use? lstm-compress, nncp, third-party library (which?), your implementation, ...?
    Do you use it as a predictor or a mixer?
    Only 25 nodes? It's fast but scarce.

  15. #640
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,019
    Thanks
    615
    Thanked 416 Times in 314 Posts
    @kaitz -> According to v67 version - for option -s15 and for some files (G.EXE, H.EXE from my filetest and AcroRd32.exe, FlashMX.pdf, MSO97.DLL from Maximum Compression) program didn't finish the compression process and exit w/o crash...

    Question about WAV model -could you implement also newest model from paq8px v179/v181 version?

    And the maybe a tip - if you want to optimize ebnwik scores then maybe it's reasoneble to look at Paq8pxd_v48_bwt4 version - it got 16'183'xxx bytes for -s8 option on the model from paq8pxd v48 version.

    And scores for paq8pxd v68 version -> 0.29% of improvement = -30KB! Very nice gain, especially for the biggest files.
    That means paq8pxd v68 got total score for my testset better than cmix v18 - 14KB less!
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	paq8pxd_v68.jpg 
Views:	50 
Size:	640.0 KB 
ID:	6797  
    Last edited by Darek; 16th August 2019 at 12:54.

  16. #641
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,019
    Thanks
    615
    Thanked 416 Times in 314 Posts
    Scores of 4 corpuses for paq8pxd v68.

    - Canterbury Corpus got the best score for paq8pxd version but other corpuses not.
    - Calgary loses 700 bytes to best v62 score.
    - MaximumCompression loses 12KB to v62 version.
    - The biggest lose to v62 version becames from Silesia corpus = 83KB (0.28%) - moslty on Mozilla (27KB), oofice (15KB) and Webster (31KB) files.

    Similarly to v67 version this version have the same issue - for option -s15 and for some files (G.EXE, H.EXE from my filetest and AcroRd32.exe, FlashMX.pdf, MSO97.DLL from Maximum Compression) program didn't finish the compression process and exit w/o crash... then for Maximum compression these three files (AcroRd32.exe, FlashMX.pdf, MSO97.DLL) are compressed with -s14 option.
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	paq8pxd_v68_4_Corpuses.jpg 
Views:	51 
Size:	1.83 MB 
ID:	6799  
    Last edited by Darek; 19th August 2019 at 22:07.

  17. Thanks:

    kaitz (26th August 2019)

  18. #642
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,019
    Thanks
    615
    Thanked 416 Times in 314 Posts
    Scores of enwik8/9 compressed by paq8pxd v68 - nice gain (0.08-0.11%), however still about 0,3% worse scores than v63 version.

    16'309'012 - enwik8 -s8 by Paq8pxd_v63, time 9'240s
    15'967'201 - enwik8 -s15 by Paq8pxd_v63, time 8'880s
    16'63'7302 - enwik8.drt -s15 by Paq8pxd_v63, time 11'837s
    126'597'584 - enwik9_1423 -s15 by Paq8pxd_v63, time 98'387s


    16'374'223 - enwik8 -s8 by Paq8pxd_v67_AVX2, +0.40% to v63score, time 6'431s,
    16'048'070 - enwik8 -s15 by Paq8pxd_v67_AVX2, +0.51% to v63 score, time 6'643s
    16'774'998 - enwik8.drt -s15 by Paq8pxd_v67_AVX2, +0.83% to v63 score, time 8'413s
    127'063'602 - enwik9_1423 -s15 by Paq8pxd_v67_AVX2, +0.37% to v63 score, time 66'041s

    16'364'165 - enwik8 -s8 by Paq8pxd_v68_AVX2
    16'033'591 - enwik8 -s15 by Paq8pxd_v68_AVX2
    16'755'942 - enwik8.drt -s15 by Paq8pxd_v68_AVX2
    126'958'003 - enwik9_1423 -s15 by Paq8pxd_v68_AVX2

  19. Thanks:

    kaitz (26th August 2019)

  20. #643
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    465
    Thanks
    193
    Thanked 289 Times in 157 Posts
    Quote Originally Posted by Darek View Post
    Scores of enwik8/9 compressed by paq8pxd v68 - nice gain (0.08-0.11%), however still about 0,3% worse scores than v63 version.

    126'958'003 - enwik9_1423 -s15 by Paq8pxd_v68_AVX2
    Did not expect that much on enwik9.
    Will try to find out where errors come on -s15. Maybe on weekend. I was on hurry when releasing so sorry about that.
    Quote Originally Posted by Mauro Vezzosi View Post
    > lstm_(25, 3, 20, 0.05)
    What implementation do you use? lstm-compress, nncp, third-party library (which?), your implementation, ...?
    Do you use it as a predictor or a mixer?
    Only 25 nodes? It's fast but scarce.
    File attached. Used this i guess https://github.com/byronknoll/lstm-compress
    It was 2 years ago when i first tested.

    And this is added to main file.


    #include "lstm.inc"
    class lstmModel1: public Model {
    BlockData& x;
    Buf& buf;
    ByteModel lstm_;
    public:
    lstmModel1(BlockData& bd,U32 val=0):x(bd),buf(bd.buf),lstm_(25, 3, 20, 0.05){ // num_cells, num_layers, horizon, learning_rate)
    srand(0xDEADBEEF);
    }
    int inputs() {return 1;}
    int p(Mixer& m,int val1=0,int val2=0){
    lstm_.Perceive(x.y);
    m.add(stretch(lstm_.Predict()));
    return 0;
    }
    virtual ~lstmModel1(){ }
    };
    Attached Files Attached Files
    KZo


  21. Thanks:

    Mauro Vezzosi (26th August 2019)

  22. #644
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    465
    Thanks
    193
    Thanked 289 Times in 157 Posts
    Made short test with above example. Used FMA compile option. Code size was smaller, assembly output showed lots of vfmadd..... Compession was better a. 30 bytes on enwik6 (internal pecision of fma on float). No speed difference, mabye 1 sec.
    Archive Not identical vs no FMA
    KZo


  23. #645
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    465
    Thanks
    193
    Thanked 289 Times in 157 Posts
    Source only paq8pxd_v69, i will not provide binary. It will compress but has probably same problems as v67 and v68 with option -15

    Code:
    Online wrt is present and fixed, now actually decodes word if code is larger then 1 byte
    WordModel form px182
        this one is not active in any models, you have to manually add it in source, replaced in model array with blankmodel
        onlyne wrt is needed so one can detect presence of BT and ET for pdf mode, had no luck to get compression working better (sort of)
    Combined models into arrays to ease mixer inputs/sets/context clutter, led models to return usage
    Colored console (windows only!), based on type stream.
    Click image for larger version. 

Name:	colored.pxd69.png 
Views:	81 
Size:	142.2 KB 
ID:	7016
    Attached Files Attached Files
    KZo


  24. Thanks (4):

    Darek (24th October 2019),Gotty (24th October 2019),moisesmcardona (23rd October 2019),Stephan Busch (23rd October 2019)

  25. #646
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,019
    Thanks
    615
    Thanked 416 Times in 314 Posts
    Could you please made some binary for windows 64?

  26. #647
    Member
    Join Date
    Jun 2009
    Location
    Puerto Rico
    Posts
    190
    Thanks
    89
    Thanked 18 Times in 14 Posts
    I compiled it with mingw.
    Attached Files Attached Files

  27. Thanks:

    Darek (24th October 2019)

  28. #648
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,019
    Thanks
    615
    Thanked 416 Times in 314 Posts
    Here are scores of paq8pxd v69 for my testset. Some nice gains for the biggest files.
    Colored console => very nice improvement!
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	paq8pxd_v69.jpg 
Views:	41 
Size:	610.7 KB 
ID:	7020  

  29. #649
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,019
    Thanks
    615
    Thanked 416 Times in 314 Posts
    Here are scores of 4 Corpuses by paq8pxd v69. Compression differences are similar to my testset, however there are some issues:

    1) still there are error which quit w/o crashing on AcroRd32.exe, MSO97.DLL files (from Maximum Compression) and G.EXE and H.EXE from my testset on -s15 option. Using -s14 is ok.

    2) there no above issues on FlashMX.pdf (Maximum Compression) and mozilla and samba files (Silesia) but for these files didn't work properly (I guess) preprocessing and parsering process and then these files have much worse compression - similar to files w/o tar files parser.
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	paq8pxd_v69_4_Corpuses.jpg 
Views:	33 
Size:	1.88 MB 
ID:	7021  

  30. Thanks:

    kaitz (25th October 2019)

  31. #650
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    465
    Thanks
    193
    Thanked 289 Times in 157 Posts
    Deflate disabled probably or some other testing thing...
    KZo


  32. #651
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,019
    Thanks
    615
    Thanked 416 Times in 314 Posts
    some enwik scores for paq8pxd:

    16'369'714 - enwik8 -s8 by Paq8pxd_v54
    16'026'652 - enwik8 -s15 by Paq8pxd_v54
    15'956'319 - enwik8.drt -s15 by Paq8pxd_v54 -> best overall score on enwik8 for paq8pxd
    126'821'031 - enwik9_1423 -s15 by Paq8pxd_v54

    starting from v57 version DRT precompressed files got much worse scores than non-precompressed

    16'309'641 - enwik8 -s8 by Paq8pxd_v61
    15'968'477 - enwik8 -s15 by Paq8pxd_v61
    16'570'543 - enwik8.drt -s15 by Paq8pxd_v61
    126'587'796 - enwik9_1423 -s15 by Paq8pxd_v61 -> best overall score on enwik9 for paq8pxd (bwt version is separated)

    16'309'012
    - enwik8 -s8 by Paq8pxd_v63 -> best overall score on enwik8 with -s8 option for paq8pxd
    15'967'201 - enwik8 -s15 by Paq8pxd_v63 -> best overall score on enwik8 (w/o DRT precompression) for paq8pxd
    16'637'302 - enwik8.drt -s15 by Paq8pxd_v63
    126'597'584 - enwik9_1423 -s15 by Paq8pxd_v63

    16'364'165 - enwik8 -s8 by Paq8pxd_v68_AVX2
    16'033'591 - enwik8 -s15 by Paq8pxd_v68_AVX2
    16'755'942 - enwik8.drt -s15 by Paq8pxd_v68_AVX2
    126'958'003 - enwik9_1423 -s15 by Paq8pxd_v68_AVX2

    16'362'891 - enwik8 -s8 by Paq8pxd_v69
    16'032'657 - enwik8 -s15 by Paq8pxd_v69
    16'697'316 - enwik8.drt -s15 by Paq8pxd_v69
    126'950'607 estimated - enwik9_1423 -s15 by Paq8pxd_v69

  33. #652
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,019
    Thanks
    615
    Thanked 416 Times in 314 Posts
    It's something wrong due enwik9 compression. I've tried both pure file and drt file and paq8pxd v69 quit in both cases (I've checked it 2 times).
    There is another strange behaviour - like on screen - big part of text of enwik9 are compressed with default part of a text.
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	paq8pxd_v69_question.jpg 
Views:	36 
Size:	128.6 KB 
ID:	7023  

  34. #653
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    465
    Thanks
    193
    Thanked 289 Times in 157 Posts
    You are using DRT processed input. Text detection was changed some versions ago so it will detect some parts as default and its ok. Same effect as with enwik8.drt files.
    If input is plain enwik9 you get (-s0):
    Click image for larger version. 

Name:	enwik9.s0.pxd69.png 
Views:	71 
Size:	18.0 KB 
ID:	7024
    KZo


  35. #654
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    465
    Thanks
    193
    Thanked 289 Times in 157 Posts
    Did small experiment to get some speed.
    StateMap has two update functions, switching comes from APM , ContextMap or ContextMap2 at pos=0xffff.
    If file/block is smaller compression time is almost same or 1-2 sec worse. Switching update probably needs longer learning time.
    It needs some more testing/tuning.
    Inline for update is pointless now.

    class StateMap {
    protected:
    const int N; // Number of contexts
    int cxt; // Context of last prediction
    void (StateMap::*update2)(int,int) ;
    Array<U32> t; // cxt -> prediction in high 22 bits, count in low 10 bits
    inline void update(int y, int limit) {
    (this->*update2)(y,limit);
    }

    void update0(int y, int limit) {
    assert(cxt>=0 && cxt<N);
    assert(y==0 || y==1);
    U32 *p=&t[cxt], p0=p[0];
    int n=p0&1023, pr=p0>>10; // count, prediction
    p0+=(n<limit);
    p0+=(((y<<22)-pr)>>3)*dt[n]&0xfffffc00;
    p[0]=p0;
    }
    void update1(int y, int limit) {
    assert(cxt>=0 && cxt<N);
    assert(y==0 || y==1);
    U32 *p=&t[cxt], p0=p[0];
    int pr=p0>>10; // prediction
    p0+=(((y<<22)-pr)>>3)*7&0xfffffc00;
    p[0]=p0;
    }
    public:
    StateMap(int n=256);
    void Reset(int Rate=0){
    update2=update0;
    for (int i=0; i<N; ++i)
    t[i]=(t[i]&0xfffffc00)|min(Rate, t[i]&0x3FF);
    }
    void switchUpdate() {
    update2=update1;
    }
    // update bit y (0..1), predict next bit in context cx
    int p(int cx, int y,int limit=1023) {
    assert(cx>=0 && cx<N);
    assert(limit>0 && limit<1024);
    assert(y==0 || y==1);
    update(y,limit);
    return t[cxt=cx]>>20;
    }
    };


    StateMap::StateMap(int n): N(n), cxt(0), t(n) {
    update2=update0;
    for (int i=0; i<N; ++i)
    t[i]=1<<31;
    }

    paq8pxd_v69sm updateted statemap
    paq8pxd_v69 statemap
    Code:
    paq8pxd_v69sm.exe -s8:2    world95.txt
    Creating archive world95.txt.paq8pxd69f with 1 file(s)...
    
    
    File list (21 bytes)
    Compressed from 21 to 23 bytes.
    Mixer time 0.0020 sec.   max inputs 1584, max contexts 74920, max context sets 31
    
    
    1/1  Filename: world95.txt (2988578 bytes)
    Block segmentation:
     0           | eoltxt   |   2988578 [0 - 2988577]
    
    
     Segment data size: 40 bytes
    
    
     TN |Type name |Count      |Total size
    -----------------------------------------
     34 |eoltxt    |         1 |   2988578
    -----------------------------------------
    Total level  0 |         1 |   2988578
    
    
    text wrt stream(9). Total 2899483
    compressed stream(12). Total 627
     Total 2899483 wrt: 1830523
    WRT dict count 1841 words.
    Mixer time 0.0390 sec.   max inputs 1584, max contexts 74920, max context sets 31
    Stream(12) compressed from 627 to 269 bytes
    WRT dict online.
     9  99.80%
     TXTWRT Count of skipped bytes 20
    Stream(9) compressed from 2899483 to 320177 bytes
    Mixer time 0.0020 sec.   max inputs 1584, max contexts 74920, max context sets 31
     Segment data compressed from 40 to 23 bytes
     Total 2988578 bytes compressed to 320542 bytes.
    Time 308.17 sec, used 3837 MB (4023977314 bytes) of memory
    
    
    paq8pxd_v69.exe -s8:2    world95.txt
    Creating archive world95.txt.paq8pxd69f with 1 file(s)...
    
    
    File list (21 bytes)
    Compressed from 21 to 23 bytes.
    Mixer time 0.0020 sec.   max inputs 1584, max contexts 74920, max context sets 31
    
    
    1/1  Filename: world95.txt (2988578 bytes)
    Block segmentation:
     0           | eoltxt   |   2988578 [0 - 2988577]
    
    
     Segment data size: 40 bytes
    
    
     TN |Type name |Count      |Total size
    -----------------------------------------
     34 |eoltxt    |         1 |   2988578
    -----------------------------------------
    Total level  0 |         1 |   2988578
    
    
    text wrt stream(9). Total 2899483
    compressed stream(12). Total 627
     Total 2899483 wrt: 1830523
    WRT dict count 1841 words.
    Mixer time 0.0320 sec.   max inputs 1584, max contexts 74920, max context sets 31
    Stream(12) compressed from 627 to 269 bytes
    WRT dict online.
     9  99.80%
     TXTWRT Count of skipped bytes 20
    Stream(9) compressed from 2899483 to 318808 bytes
    Mixer time 0.0010 sec.   max inputs 1584, max contexts 74920, max context sets 31
     Segment data compressed from 40 to 23 bytes
     Total 2988578 bytes compressed to 319173 bytes.
    Time 314.81 sec, used 3837 MB (4023977314 bytes) of memory
    KZo


  36. #655
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,760
    Thanks
    274
    Thanked 1,200 Times in 668 Posts
    Btw, when I tested counter update with "if" vs branchless ones, former actually won...
    likely because branches on data bits from compressible data are actually predictable.

  37. #656
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,760
    Thanks
    274
    Thanked 1,200 Times in 668 Posts
    I tested different compilers with paq8pxd, clang seems to win?
    Code:
     67.485s 66.734s:  pxd_cl90avx.exe -s7 1 1 / -d
     67.453s 66.828s:  pxd_gcc82avx.exe -s7 1 1 / -d
     69.235s 69.843s:  pxd_v69.exe -s7 1 1 / -d
     71.141s 71.125s:  pxd_ic19avx.exe -s7 1 1 / -d
    Attached Files Attached Files

  38. Thanks:

    kaitz (2nd November 2019)

  39. #657
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,019
    Thanks
    615
    Thanked 416 Times in 314 Posts
    Quote Originally Posted by Shelwien View Post
    I tested different compilers with paq8pxd, clang seems to win?
    Code:
     67.485s 66.734s:  pxd_cl90avx.exe -s7 1 1 / -d
     67.453s 66.828s:  pxd_gcc82avx.exe -s7 1 1 / -d
     69.235s 69.843s:  pxd_v69.exe -s7 1 1 / -d
     71.141s 71.125s:  pxd_ic19avx.exe -s7 1 1 / -d
    Time tests of my testset with -s9 option (w/o G.EXE due to program hang):

    3660s = pxd_cl90avx.exe
    3840s = pxd_gcc82avx.exe
    4020s = pxd_v69.exe
    4260s = pxd_ic19avx.exe

    Unfortunatelly cl and ic versions hang up with G.EXE file in the same moment - showed on attached screenshot.
    The second insight - ic and cl versions gives a little different outputs as in also attached table.
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	v69_hang_up.jpg 
Views:	38 
Size:	103.4 KB 
ID:	7026   Click image for larger version. 

Name:	v69_compile_outputs.jpg 
Views:	41 
Size:	131.1 KB 
ID:	7027  

  40. Thanks:

    kaitz (2nd November 2019)

  41. #658
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,760
    Thanks
    274
    Thanked 1,200 Times in 668 Posts
    Different outputs are likely due to float-point (there're doubles in linearPredictionModel, im24, im8).
    No idea as to hanging.

  42. Thanks:

    kaitz (2nd November 2019)

  43. #659
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,019
    Thanks
    615
    Thanked 416 Times in 314 Posts
    Quote Originally Posted by Shelwien View Post
    Different outputs are likely due to float-point (there're doubles in linearPredictionModel, im24, im.
    No idea as to hanging.
    Decompression for all four versions gives the same files and for all sha1 checksums are Ok.
    According to hanging - it hangs only for only these two mentioned versions - ic19avx and cl90avx. The moment of hang is the moment just after decomposition of the file.

  44. #660
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    465
    Thanks
    193
    Thanked 289 Times in 157 Posts
    This is probably good point. I dont like floats and etc. This was reason i used tta for sound.
    There are probably other broblems aswel. Something i missed.

    EDIT:
    I need 16GB more memory to test.
    Last edited by kaitz; 2nd November 2019 at 02:28. Reason: More mem
    KZo


Page 22 of 26 FirstFirst ... 122021222324 ... LastLast

Similar Threads

  1. FreeArc compression suite (4x4, Tornado, REP, Delta, Dict...)
    By Bulat Ziganshin in forum Data Compression
    Replies: 554
    Last Post: 26th September 2018, 02:41
  2. Dict preprocessor
    By pat357 in forum Data Compression
    Replies: 5
    Last Post: 2nd May 2014, 21:51

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •