Page 4 of 22 FirstFirst ... 2345614 ... LastLast
Results 91 to 120 of 642

Thread: Paq8pxd dict

  1. #91
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    385
    Thanks
    142
    Thanked 213 Times in 115 Posts
    Tested on: Intel i3 4010U 1.7GHz, 4GB PC3-12800 DDR3
    Compiled 32bit AVX2.

    I will test enwik9.

    Code:
    E:\>paq8pxd_v11 -8 E:\enwik8
    Creating archive E:\enwik8.paq8pxd11 with 1 file(s)...
    
    
    File list (18 bytes)
    Compressed from 18 to 18 bytes.
    
    
    1/1  Filename: E:/enwik8 (100000000 bytes)
    Block segmentation:
     0           | utf-8     | 100000000 b [0 - 99999999](wt: 61838435)
    Compressed from 100000000 to 16600417 bytes.
    
    
    Total 100000000 bytes compressed to 16600471 bytes.
    
    
     Segment data size: 9 bytes
    
    
     TN |Type name |Count      |Total size
    -----------------------------------------
     11 |utf-8     |         1 |  100000000
    -----------------------------------------
    Total level  0 |         1 |  100000000
    
    
    Time 7861.08 sec, used 1768244214 bytes of memory
    Code:
    paq8pxd_v11 -8 enwik9
    Creating archive enwik9.paq8pxd11 with 1 file(s)...
    
    
    File list (19 bytes)
    Compressed from 19 to 18 bytes.
    
    
    1/1  Filename: enwik9 (1000000000 bytes)
    Block segmentation:
     0           | utf-8     |1000000000 b [0 - 999999999](wt: 625961245)
    Compressed from 1000000000 to 134732117 bytes.
    
    
    Total 1000000000 bytes compressed to 134732171 bytes.
    
    
     Segment data size: 9 bytes
    
    
     TN |Type name |Count      |Total size
    -----------------------------------------
     11 |utf-8     |         1 | 1000000000
    -----------------------------------------
    Total level  0 |         1 | 1000000000
    
    
    Time 77121.74 sec, used 1768244217 bytes of memory
    Last edited by kaitz; 13th July 2014 at 12:31. Reason: enwik9
    KZo


  2. #92
    Member
    Join Date
    Jun 2014
    Location
    Ro
    Posts
    20
    Thanks
    4
    Thanked 4 Times in 4 Posts
    I modified the sourcecode for the paq8pxd_v11 in order to compile it for 64bit, with 64bit pointers, and created a version of the software with a -10 compression option. I will not test decompression because it took a fairly long time for compression and I was only interested to see how the output goes in size. The resulting file is 128.147.456 bytes. Again, this is an unofficial test with untested results. If anyone wants to try this, I can post the modified sourcecode, even compile it for some processor or specific setting for Windows OS, just let me know.

  3. The Following User Says Thank You to AlexDoro For This Useful Post:

    Skymmer (13th July 2014)

  4. #93
    Member Skymmer's Avatar
    Join Date
    Mar 2009
    Location
    Russia
    Posts
    681
    Thanks
    37
    Thanked 168 Times in 84 Posts
    Impressive! I woild like to test it so it will be nice if you'll post the source.

  5. #94
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    778
    Thanks
    63
    Thanked 273 Times in 191 Posts
    Quote Originally Posted by AlexDoro View Post
    even compile it for some processor or specific setting for Windows OS, just let me know.
    I can test till 128GB memory use, option -14? I like to test -11 or higher if you can provide .exe for Windows 2012 R2/8.1 for modern CPU's.

  6. #95
    Member
    Join Date
    Jun 2014
    Location
    Ro
    Posts
    20
    Thanks
    4
    Thanked 4 Times in 4 Posts
    Good news and bad news. The good news are that I post the source and the compiled exe. The compiler options are -msse2 -Ofast -m64 -g3 -static-libgcc (and some included libs). If you want to use the -10 option, use the default compression (no param like -10). You can use the -9 option if you want. Btw, the array bounds check options is set and if you want it off, add the NDEBUG define back.
    The bad news. The source is not ready for a -11 option or higher, I will try to do this (it is difficult), but I won't be able to test it since I only have 8GB. The major step in this process is being able to switch the (unsigned) integer indexing to long indexing. I hope I will be able to do it.
    One of my initial thoughts was to port the PAQ implementation to a higher level language, so users can add context models or other predictors more easily. I still have troubles in porting the pointer tricks. I know the speed gain from them is important, but I am in favor of clean and easy to read code (code as documentation).

    Edit:
    More bad news, when decompressing enwik6 (with -10 option), I get the "Segment data corrupted." error. My guess is that the number 10 written in the header wasn't supposed to be a two character string. The -9 option works. I will try to fix this.

    Edit2: added fix to overcome the above issue.
    Attached Files Attached Files
    Last edited by AlexDoro; 13th July 2014 at 18:13. Reason: Decompressing with -10

  7. The Following User Says Thank You to AlexDoro For This Useful Post:

    Sportman (14th July 2014)

  8. #96
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    385
    Thanks
    142
    Thanked 213 Times in 115 Posts
    Quote Originally Posted by AlexDoro View Post
    Btw, the array bounds check options is set and if you want it off, add the NDEBUG define back.

    Removing this makes paq slow and should be used only when debugging.

    Quote Originally Posted by AlexDoro View Post
    One of my initial thoughts was to port the PAQ implementation to a higher level language, so users can add context models or other predictors more easily. I still have troubles in porting the pointer tricks. I know the speed gain from them is important, but I am in favor of clean and easy to read code (code as documentation).
    What kind of language?

    ContextMap can only support up to 4GB memory, so if all models use maximum, its about 24-32GB memory that can be used (dicttxt). Correct me if i am wrong.
    KZo


  9. #97
    Member
    Join Date
    Jun 2014
    Location
    Ro
    Posts
    20
    Thanks
    4
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by kaitz View Post
    What kind of language?
    I was thinking about c sharp. The linux users have mono, so there isn't a big loss in the community if this step is made. I started with the predictor part of the pxd_v7 implementation. Theoretically, I translated the whole part, The problem is, it does not yield the same accuracy (or not an expected one anyway). My guess is that I failed at some typecasts (c sharp strongly types anything) and this lead to algorithm loss. The prediction works good but with a loss in prediction power (it's strange). Do you know any good C++ debug tools which would help me check the integrity of some steps? I mean like a step by step debugger, preferably in an integrated environment. The dev-cpp environment is very bad at debugging, can't get my debugging in it in any way. Debugging a paq algorithm is hard enough already (as you obviously know).

  10. #98
    Member
    Join Date
    Jun 2009
    Location
    Kraków, Poland
    Posts
    1,474
    Thanks
    26
    Thanked 121 Times in 95 Posts
    I've developed C/ C++ in NetBeans and debugging works reasonably there.

  11. #99
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    385
    Thanks
    142
    Thanked 213 Times in 115 Posts
    I agree that dev-cpp has problems and it crashes, still works for me. I use sometimes MS Visual C++ 2010 express. While back i used Code::Blocks (cant remember if it had debugging support).

    As for C#, i think it will be slower.
    KZo


  12. #100
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 660 Times in 354 Posts
    1. there are lots of mobile/web paltforms these days, so C# isn't a portable solution
    2. you can install ms visual studio express for free

  13. #101
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    778
    Thanks
    63
    Thanked 273 Times in 191 Posts
    Visual Studio 14 CTP 2 you can also download and "test" for free (CTP 1 needed VS free system or VM, did not tested CTP 2 install yet):
    http://www.visualstudio.com/en-us/do...udio-14-ctp-vs

    Microsoft is working at "C++ performance with the managed-code benefits of C#"
    http://blogs.msdn.com/b/dotnet/archi...e-preview.aspx
    Looks like you can also compile for mobile ARM CPU's.

  14. #102
    Member
    Join Date
    Jun 2008
    Location
    G
    Posts
    372
    Thanks
    26
    Thanked 22 Times in 15 Posts
    I think big architecture changes might not make sense, because paq8 has no future except being an experimental compressor. In my opinion it would make more sense to do this to zpaq, if at all. Because zpaq has during the forward & backward archive compartibily all chances to be a very succesful long living archiver. Nevertheless in my opinion changes to improve paq8 performance shoushould be done.

  15. #103
    Member
    Join Date
    Jun 2014
    Location
    Ro
    Posts
    20
    Thanks
    4
    Thanked 4 Times in 4 Posts
    @thometal: I'm not sure if this is the right thread for a debate like this, but I'm going to post my opinion here. Paq8 has a future as long as people choose to invest time in it. Matt still maintains zpaq (and probably will for a long time) and I'm sure he has the roadmap for it. Paq8 has the goal to search for limits in online prediction power. There were no fundamental changes in it's structure. The most changes kaitz made were to adapt the domain specific knowledge to certain types of data (text, image, audio, exe), like detecting which type of data one handles, for text dynamic dictionary preprocessing, or improved image models based on well known (mathematically proven) predictors, and why not, speed improvements. In order to see if the predictor behave well in time, you need large test data, therefore speed was an important factor to still keep the interest in paq8. I'm sure there are other people, like me, who would like to try to improve the existing state of the art. The current source code, while having speed and memory consumption in focus, is hard to grasp at first sight and might discourage fresh minds trying to improve it.
    If you give up on just a few cases on pointer arithmetic or magic numbers, the whole code becomes a lot easier to read. Going further, if you separate the concerns in distinct classes or definitions, it becomes more clear where one block ends and one begins. Compiling this will lead to a slower code, because it is closer to the human language than to machine language, but expressing ideas suddenly is accessible to more than just veteran programmers alone. Implementing higher level concepts like NLP is very difficult if you have to go many steps down towards the machine language.
    My personal belief is that there still is a lot of potential in this field. For instance, Byron Knoll tackled the problem but used the entire paq predictor as an input to his context mixer. Many others might benefit from this knowledge if they could grasp the concepts behind it.

  16. #104
    Member
    Join Date
    Jun 2008
    Location
    G
    Posts
    372
    Thanks
    26
    Thanked 22 Times in 15 Posts
    oh so i missunderstood you i though you want to make paq8 an archiver but of course research and clean and easy understandable code is very important, i fully agree with you.

  17. #105
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    778
    Thanks
    63
    Thanked 273 Times in 191 Posts
    Quote Originally Posted by AlexDoro View Post
    The good news are that I post the source and the compiled exe.
    The bad news. The source is not ready for a -11 option or higher
    I only tried -11 so far at a big IIS log file:

    paq8pxd_v11 -8 (original):
    Total 264996083 bytes compressed to 3311151 bytes.
    Time 7511.24 sec, used 1768244199 bytes of memory

    paq8pxd_v11 -11 (AD modified):
    Total 264996083 bytes compressed to 3267555 bytes.
    Time 9298.61 sec, used 852426744 bytes of memory

  18. #106
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts
    You can do things in PAQ that you can't in ZPAQ because you aren't worried about compatibility between versions. PAQ is still a valuable research tool for discovering new techniques that might eventually work their way into a future ZPAQ version.

  19. #107
    Member
    Join Date
    Jun 2014
    Location
    Ro
    Posts
    20
    Thanks
    4
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by Sportman View Post
    I only tried -11 so far at a big IIS log file:
    -11 doesn't mean anything, it goes to the default option which is -10. Valid option for command line are from -0 to -9. The displayed "used memory" doesn't work because it overflows, please check task manager.
    But yeah, the improvement is clear.

  20. #108
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    385
    Thanks
    142
    Thanked 213 Times in 115 Posts
    I wanted to improve speed in ContextMap using SIMD. So i created partial SSE2 and SSSE3 version of get function. Im not sure if priority part can be done.

    Timings in seconds
    Code:
    -4       enwik6    ooffice
    none     113.88    853.19
    SSE2     112.19    836.49
    SSSE3    111.83    836.68
    To test replace ContextMap get with this (VC and GCC version):
    #ifdef _MSC_VER
    #include <intrin.h>
    U32 __inline clz(U32 value){ //asume newer version of vc
    unsigned long leading_zero = 0;
    if (_BitScanReverse( &leading_zero, value)) return 31-leading_zero;
    else return 32;
    }
    #endif
    #define SIMD_GET_SSE
    // Find or create hash element matching checksum ch
    inline U8* ContextMap::E::get(U16 ch) {

    if (chk[last&15]==ch) return &bh[last&15][0];
    int b=0xffff, bi=0;
    #if defined(SIMD_GET_SSE)
    const __m128i xmmch=_mm_set1_epi16 (short(ch)); //fill 8 ch values
    __m128i tmp=_mm_load_si128 ((__m128i *) &chk[0]); //load 8 values (8th will be discarded)
    #if defined(__SSE2__)
    tmp=_mm_shufflelo_epi16(tmp,0x1B); //swap order for mask (0,1,2,3)
    tmp=_mm_shufflehi_epi16(tmp,0x1B); //(0,1,2,3)
    tmp=_mm_shuffle_epi32(tmp,0x4E); //(1,0,3,2)
    #elif defined(__SSSE3__)
    #include <immintrin.h>
    const __m128i vm=_mm_setr_epi8(14,15,12,13,10,11,8,9,6,7,4,5,2,3,0,1);// initialise vector mask
    tmp=_mm_shuffle_epi8(tmp,vm);
    #endif
    tmp=_mm_cmpeq_epi16 (tmp,xmmch); //compare ch values
    tmp=_mm_packs_epi16(tmp,_mm_setzero_si128()); //pack result
    U32 t=(_mm_movemask_epi8(tmp))>>1; //get mask of comparsion, bit is set if eq, discard 8th bit
    #if defined(_MSC_VER)
    U32 a; //index into bh or 7 if not found
    if(t){
    a=(clz(t)-1)&7;
    return last=last<<4|a, (U8*)&bh[a][0];
    }
    else a=7;
    #elif ((__GNUC__ >= 4) || ((__GNUC__ == 3) && (__GNUC_MINOR__ >= 4)))
    U32 a;
    if(t){
    a=(__builtin_clz(t)-1)&7;
    return last=last<<4|a,(U8*)&bh[a][0];}
    else a=7;
    #endif


    for (int i=0; i<7; ++i) {
    int pri=bh[i][0];
    if (pri<b && (last&15)!=i && last>>4!=i) b=pri, bi=i;
    }
    return last=0xf0|bi, chk[bi]=ch, (U8*)memset(&bh[bi][0], 0, 7);
    #else
    for (int i=0; i<7; ++i) {
    if (chk[i]==ch) return last=last<<4|i, (U8*)&bh[i][0];
    int pri=bh[i][0];
    if (pri<b && (last&15)!=i && last>>4!=i) b=pri, bi=i;
    }
    return last=0xf0|bi, chk[bi]=ch, (U8*)memset(&bh[bi][0], 0, 7);
    #endif
    }
    KZo


  21. #109
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    385
    Thanks
    142
    Thanked 213 Times in 115 Posts
    PAQ8PXD_V12
    DIFFERENCES FROM PAQ8PXD_V11

    • -increase SZDD & SZ size (16MB uncompressed)
    • -change normalmodel, recordmodel1
    • -recordmodel,wavmodel only when needed (smaller memory usage)
    • -wavmodel uses separate recordmodel
    • -another change in 8-bit image model
    • -wordmodel memory increase, dmcmodel memory decrease (maintain same memory usage)
    • -small fixes
    • -SSE2/SSSE3 version of ContextMap::Get (By default not used)
    • reduce warnings at compile time


    8-bit image model is faster and maintains same compression level on most files i have tested. Needs more work on artificial images. This can be probably tested on detection phase.
    Like this: abs((buf(2)+buf(w+1)-buf(w)) -(buf(w-1)+buf(1)-buf(w)))>255
    Count all values in image above threshold (255) and all below. If below is almost or more then 1/3 of above then we should use paq8pxd_v11 model. At least i think so . I left code in so above can be tested on single image files. You need to uncomment some lines to get above values.

    I have included full attempt of ContextMap::Get SIMD version. First part is faster (to test uncomment line #define SIMD_GET_SSE). There is code for full version witch is same speed but uses SSE2 code. I'm no SIMD guru, at least it works.

    enwik8 compresses about 25000 bytes better then previous paq8pxd versions.

    EDIT:

    Left image is original. Next is below and last is above
    Click image for larger version. 

Name:	image002.png 
Views:	317 
Size:	150.9 KB 
ID:	3067Click image for larger version. 

Name:	image002 b.png 
Views:	322 
Size:	148.3 KB 
ID:	3068Click image for larger version. 

Name:	image002 a.png 
Views:	314 
Size:	11.2 KB 
ID:	3069

    Left image is original. Next is below and last is above
    Click image for larger version. 

Name:	image006.png 
Views:	320 
Size:	312.5 KB 
ID:	3070Click image for larger version. 

Name:	image006 b.png 
Views:	319 
Size:	477.5 KB 
ID:	3071Click image for larger version. 

Name:	image006 a.png 
Views:	287 
Size:	323.8 KB 
ID:	3072

    Images are created like this:
    value=abs((buf(2)+buf(w+1)-buf(w)) -(buf(w-1)+buf(1)-buf(w)))
    below
    if value>255 output 0
    else value

    above
    if value>255 output value>>8
    else 255
    Attached Files Attached Files
    Last edited by kaitz; 30th July 2014 at 13:09.
    KZo


  22. #110
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts
    LTCB results. http://mattmahoney.net/dc/text.html#1345

    Edit: updated Silesia. Decompression checks OK. http://mattmahoney.net/dc/silesia.html
    Last edited by Matt Mahoney; 1st August 2014 at 18:08.

  23. #111
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    873
    Thanks
    462
    Thanked 175 Times in 85 Posts
    Dear Kaido,

    I have finally managed to test paq8pxd_12 using option 7.
    There were problems in detection of WAV, PPM, PGM and UTF-8.
    The whole log can be found here: http://www.squeezechart.com/paq8pxd_v12.log
    I will post SqueezeChart results later today.

  24. #112
    Member
    Join Date
    May 2008
    Location
    brazil
    Posts
    163
    Thanks
    0
    Thanked 3 Times in 3 Posts
    Is it possible to separate the detection part of paq from compression part?

  25. #113
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    385
    Thanks
    142
    Thanked 213 Times in 115 Posts
    Quote Originally Posted by Stephan Busch View Post
    Dear Kaido,

    I have finally managed to test paq8pxd_12 using option 7.
    There were problems in detection of WAV, PPM, PGM and UTF-8.
    The whole log can be found here: http://www.squeezechart.com/paq8pxd_v12.log
    I will post SqueezeChart results later today.
    PPM,PGM, is problematic. For single files there is no problem. If image header is not at block start (hardcoded) then it fails detection and generates UTF-8 blocks or whatever it detects. There are lots of file that generate false header detection without actual image data.
    I was looking some ways to confirm if data is actually image. Using fourier transform - to get most likely periodicity. But I'm not very good at this thing.
    KZo


  26. #114
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    873
    Thanks
    462
    Thanked 175 Times in 85 Posts
    Maybe it would help to let Paq8pxd accept wildcards and recursion so that there's no need to use TAR anymore

  27. #115
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    385
    Thanks
    142
    Thanked 213 Times in 115 Posts
    All _pxd version should be compiled with wildcard support. Drop folder to it and if more then one file is listed then it works.

    I usually use .bat file, something like this:
    Code:
    paq8pxd_v12 -8 %1
    pause
    On Windows files/folder must be on same location as executable.

    Other way is set environment variable PATH to location of compressor. Can't remember where archive will be created then.
    KZo


  28. #116
    Member Skymmer's Avatar
    Join Date
    Mar 2009
    Location
    Russia
    Posts
    681
    Thanks
    37
    Thanked 168 Times in 84 Posts
    A couple of months ago I started to play with compiling. Basicly due the fact, that its really disappointing when somebody release some project as source-only and you need to wait for some good person, who will be kind enough to make a binary. At some point I realized that compiling is a kinda attractive process, so I decided to see how much I can improve the paq8pxd speed.
    And I must say that I never thought that there are so much nuances in compilers.
    Good side effect of my attempts is that I briefly started to understand the C++ code and now it looks for me not as some strange lines
    But back to deal... With a big pleasure I would like to present you the PAQ8pxd_v12-skbuild-1. Its a set of alternative compiles with the following goodies:

    Code:
    - Speed optimized compiles for Windows platform (speedup depends on content, 12.5% faster on my set, 10.6% faster on enwik7)
    - Compiled with SIMD_GET_SSE (0.8% faster than without on my set)
    - Both x86 and x64 builds provided
    - Support for unofficial -9 and -10 compression levels (-10 available in x64 build only)
    - Redesigned console help:
      - correct values of memory requirements for every level
      - levels shown in a different manner
      - added examples of directory compression
      - a little bit more (surprise!)
    Some tests...
    Code:
    			TestBed_2_Full		enwik7
    			--------------		-------
    original -8		635.718			358.218
    skbuild-1-x86 -8	585.718 (7.9%)		336.203 (6.1%)
    skbuild-1-x64 -8	556.421	(12.5%)		320.328 (10.6%)
    
    SIMD_GET_SSE
    ------------
    default			560.796s
    SIMD_GET_SSE		556.421s
    There is no change in models so output archives must be identical. But..... but there is a problem - x64 compile gives different output file comparing to x86, while original EXE and skbuild-1-x86 give identical output. More strangely is if I pack all files from testset separately then output archives from each file are all identical. At first I thought that maybe its due the TAR's structure, but no - same ***t happens with 7z store container. I also tried to turn off optimizations and change floating point calculation options but with no luck.
    Anyway, x64 version can decompress its own created archives, so at this moment please consider x64 build as experimental There are a lot of things to do, uncluding trials with Intel Compiler and ported inline assembly, so probably this version is not last.
    Attached Files Attached Files

  29. The Following 5 Users Say Thank You to Skymmer For This Useful Post:

    kaitz (17th August 2014),Matt Mahoney (10th August 2014),Samuraikarte (9th August 2014),Sportman (13th August 2014),Stephan Busch (13th August 2014)

  30. #117
    Member
    Join Date
    Jun 2008
    Location
    G
    Posts
    372
    Thanks
    26
    Thanked 22 Times in 15 Posts
    Has paq8pxd m/t support?

  31. #118
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    778
    Thanks
    63
    Thanked 273 Times in 191 Posts
    Quote Originally Posted by Skymmer View Post
    PAQ8pxd_v12-skbuild-1
    enwik8 - 16,372,331 bytes, 2826.75 sec., 64bit -10
    enwik9 - 129,827,930 bytes, 28313.259 sec., 64bit -10

    enwik9 + prog:
    129,827,930 bytes + 422,400 bytes = 130,250,330 bytes (1.63% improvement compared paq8hp12any)

    paq8hp12any enwik9 + prog:
    132,045,026 bytes + 330,700 bytes = 132,375,726 bytes
    Last edited by Sportman; 14th August 2014 at 11:37. Reason: Added enwik9

  32. #119
    Member
    Join Date
    Jun 2014
    Location
    Ro
    Posts
    20
    Thanks
    4
    Thanked 4 Times in 4 Posts
    What is interesting about the "unofficial -9 and -10 compression levels" is that they do not guarantee a better compression ratio. For instance, if you compress the silesia corpus with option -9, you get an overall worse ratio. If you look at files individually, you see that text related ones get compressed better while images and others get compressed worse. I believe that lower memory settings throw away some statistics that affect the performance of the logistic loss. What if -8 level is too high anyway for some of these? What do you think?

  33. #120
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    873
    Thanks
    462
    Thanked 175 Times in 85 Posts
    I am testing PAQ8pxd_v12-skbuild-1 using option -8 with drag & drop.
    File type detection works - but the one wav file of the Audio testset still doesn't get recognized while previous versions (v3) recognized it.
    This is the file : http://www.squeezechart.com/rock.wav

Page 4 of 22 FirstFirst ... 2345614 ... LastLast

Similar Threads

  1. FreeArc compression suite (4x4, Tornado, REP, Delta, Dict...)
    By Bulat Ziganshin in forum Data Compression
    Replies: 554
    Last Post: 26th September 2018, 02:41
  2. Dict preprocessor
    By pat357 in forum Data Compression
    Replies: 5
    Last Post: 2nd May 2014, 21:51

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •