Page 14 of 14 FirstFirst ... 4121314
Results 391 to 419 of 419

Thread: cmix

  1. #391
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    962
    Thanks
    573
    Thanked 397 Times in 295 Posts
    Yes. Sorry for mistake.
    From other hand I'm still testing rnn for cmv for first three files.
    Rsults are interesting but it's still few days to finish.

  2. #392
    Member
    Join Date
    Jun 2009
    Location
    Puerto Rico
    Posts
    184
    Thanks
    80
    Thanked 16 Times in 12 Posts
    Can someone compile the cmix v17 build for Windows? https://github.com/byronknoll/cmix/releases/tag/v17

  3. #393
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,423
    Thanks
    223
    Thanked 1,053 Times in 565 Posts
    Build target x64/SSE4. Not tested. IntelC19 and gcc82/mingw.
    Attached Files Attached Files

  4. Thanks (3):

    CompressMaster (29th April 2019),moisesmcardona (7th April 2019),xinix (7th April 2019)

  5. #394
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    962
    Thanks
    573
    Thanked 397 Times in 295 Posts
    Quote Originally Posted by moisesmcardona View Post
    Can someone compile the cmix v17 build for Windows? https://github.com/byronknoll/cmix/releases/tag/v17
    There is Byron's Windows compile on official cmix page and it works fine: http://www.byronknoll.com/cmix.html

  6. Thanks (2):

    CompressMaster (29th April 2019),moisesmcardona (7th April 2019)

  7. #395
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    962
    Thanks
    573
    Thanked 397 Times in 295 Posts
    Byron - are yours test scores on official cmix page made with new dictionary file included in cmix_v17 package?

  8. #396
    Member
    Join Date
    Mar 2011
    Location
    USA
    Posts
    232
    Thanks
    109
    Thanked 112 Times in 68 Posts
    Quote Originally Posted by Darek View Post
    Byron - are yours test scores on official cmix page made with new dictionary file included in cmix_v17 package?
    Yes.

  9. Thanks:

    Darek (8th April 2019)

  10. #397
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    962
    Thanks
    573
    Thanked 397 Times in 295 Posts
    Here are scores of my testset for cmix v17 vs. latest cmix v16f. In total there is a 8450 bytes loss, mainly for biggest, non-textual files.
    Pure textual files got very nice gain = 0.35% which is impressive.

    p.s. scores aren't optimal yet - I've based on v16e/f optimal options.
    I'm testing now other options (there are four: 1) just -c, 2) -c english.dic, 3) -s english.dic then -c and 4) -s english.dic then -c english.dic).

    I'll try to test v17 with older dictionary (from v16).
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	cmix_v17.jpg 
Views:	46 
Size:	618.0 KB 
ID:	6563  
    Last edited by Darek; 8th April 2019 at 04:21.

  11. #398
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    962
    Thanks
    573
    Thanked 397 Times in 295 Posts
    Quote Originally Posted by Shelwien View Post
    Build target x64/SSE4. Not tested. IntelC19 and gcc82/mingw.
    IC19 version doesn't work on my system.

    Comparison of Byron's compile and gcc82 ( just -c option):
    994350 bytes -> 139648 bytes in 1636.45 s. - Byron compile
    994350 bytes -> 139799 bytes in 2737.96 s. - gcc82 compile
    Last edited by Darek; 8th April 2019 at 04:21.

  12. #399
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    962
    Thanks
    573
    Thanked 397 Times in 295 Posts
    Final scores of my testset for cmix v17 - looks that this version have much different characteristics than v16f - different options got best scores.

    @mpais - request - could you add LZW transform from paq8px v178 for TGA and TIFF files to cmix? Is it possible?
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	cmix_v17.jpg 
Views:	58 
Size:	617.6 KB 
ID:	6568  

  13. #400
    Member
    Join Date
    Sep 2015
    Location
    Italy
    Posts
    250
    Thanks
    109
    Thanked 146 Times in 107 Posts
    Configurable adam learning rate, 2019/04/14:
    - Why didn't you write directly learning_rate instead of learning_rate * 0.067?
    - Will you use learning_rate statically (a fixed value) or dynamically (a value that changes during compression)?
    - How about configuring also sqrt(5e-5 * t + 1)?
    - In my tests sometimes beta1 = 0.7/0.8 is better than 0.9.
    - https://bellard.org/nncp/nncp.pdf, 2.3 Training details: "We use the Adam optimizer [5] with beta1 = 0, beta2 = 0.9999 and eps = 10^-5. No gradient clipping is done.".

  14. #401
    Member
    Join Date
    Mar 2011
    Location
    USA
    Posts
    232
    Thanks
    109
    Thanked 112 Times in 68 Posts
    Quote Originally Posted by Mauro Vezzosi View Post
    Configurable adam learning rate, 2019/04/14:
    - Why didn't you write directly learning_rate instead of learning_rate * 0.067?
    cmix actually uses two optimizers: gradient descent for the final softmax layer, and adam for everything else. My LSTM implementation takes a single learning rate parameter as input, which was used for the gradient descent (the adam parameters were just hardcoded in place to avoid too many configurable parameters). The latest commit is mostly refactoring: it uses the same learning rate parameter for both gradient descent and adam (with a multiplier in adam to make it close to the original hardcoded value).

    Quote Originally Posted by Mauro Vezzosi View Post
    - Will you use learning_rate statically (a fixed value) or dynamically (a value that changes during compression)?
    "learning_rate_" is a fixed value (but "alpha" has the sqrt(5e-5 * t + 1) decay).

    Quote Originally Posted by Mauro Vezzosi View Post
    - How about configuring also sqrt(5e-5 * t + 1)?
    I guess that could also be made into a configurable parameter - but I also want to avoid adding too many parameters to reduce complexity (i.e. only expose parameters that would make a significant difference when tuning for some new data set).

    Quote Originally Posted by Mauro Vezzosi View Post
    - In my tests sometimes beta1 = 0.7/0.8 is better than 0.9.
    - https://bellard.org/nncp/nncp.pdf, 2.3 Training details: "We use the Adam optimizer [5] with beta1 = 0, beta2 = 0.9999 and eps = 10^-5. No gradient clipping is done.".
    Yeah, I will continue experimenting with tuning these parameters. In NNCP the note is also added: "In our implementation, eps is added to the running average before taking the square root"

  15. #402
    Member
    Join Date
    Mar 2011
    Location
    USA
    Posts
    232
    Thanks
    109
    Thanked 112 Times in 68 Posts
    Quote Originally Posted by Mauro Vezzosi View Post
    - https://bellard.org/nncp/nncp.pdf, 2.3 Training details: "We use the Adam optimizer [5] with beta1 = 0, beta2 = 0.9999 and eps = 10^-5
    I tried using these parameters in cmix (and moving eps into the square root). It does significantly better on enwik8:

    before: 14872664
    after: 14856010

    I'll continue doing some more parameter tuning.

  16. #403
    Member
    Join Date
    Jun 2018
    Location
    Slovakia
    Posts
    154
    Thanks
    44
    Thanked 10 Times in 10 Posts
    Stupid question, but I need to ask.
    I need to compress files with latest cmix release (v17), but drag-and-drop method doesn´t work. What´s the correct command? I´m using Windows 7.

    Thanks.
    CompressMaster

  17. #404
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,423
    Thanks
    223
    Thanked 1,053 Times in 565 Posts
    Code:
    \cmix\src>cmix.exe
    cmix version 12
    With preprocessing:
        compress:           cmix -c [dictionary] [input] [output]
        only preprocessing: cmix -s [dictionary] [input] [output]
        decompress:         cmix -d [dictionary] [input] [output]
    Without preprocessing:
        compress:   cmix -c [input] [output]
        decompress: cmix -d [input] [output]
    So either
    cmix -c english.dic input output
    or
    cmix -c input output

    Keep in mind that its good to have >32GB of RAM for it to work.

  18. Thanks:

    CompressMaster (29th April 2019)

  19. #405
    Member
    Join Date
    Jun 2018
    Location
    Slovakia
    Posts
    154
    Thanks
    44
    Thanked 10 Times in 10 Posts
    I have encountered critical problem when I´ve tried to compress my textfile with cmix 17 (March 2019). Screenshot is attached. Seems to be an issue with memory (I have 4GB). Where´s the problem? Thanks.

    CompressMaster
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	cmix error.png 
Views:	52 
Size:	23.7 KB 
ID:	6601  

  20. #406
    Member
    Join Date
    Mar 2011
    Location
    USA
    Posts
    232
    Thanks
    109
    Thanked 112 Times in 68 Posts
    Quote Originally Posted by CompressMaster View Post
    I have encountered critical problem when I´ve tried to compress my textfile with cmix 17 (March 2019). Screenshot is attached. Seems to be an issue with memory (I have 4GB). Where´s the problem? Thanks.
    CompressMaster
    Yup, that bad_alloc error is probably due to not enough memory. 32GB of RAM is recommended to run cmix. It might be possible to run with 4GB if you have a lot of swap space, but it would be very slow.

  21. Thanks:

    CompressMaster (7th August 2019)

  22. #407
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,423
    Thanks
    223
    Thanked 1,053 Times in 565 Posts
    You can also compile it with -DDEFAULT_OPTION=4 or something.
    Also this:
      AddByteModel(new PPMD::PPMD(6, 1200, manager_.bit_context_, vocab_));
    AddByteModel(new PPMD::PPMD(16, 1200, manager_.bit_context_, vocab_));

    1200 here is allocated memory in MB, can be reduced too.
    Maybe a few other places.

    Of course, the compression would be worse than normal cmix.
    But maybe still better than paq8?

  23. #408
    Member
    Join Date
    Sep 2015
    Location
    Italy
    Posts
    250
    Thanks
    109
    Thanked 146 Times in 107 Posts
    cmix v18 2019/08/01
    http://www.byronknoll.com/cmix.html
    https://github.com/byronknoll/cmix
    https://github.com/byronknoll/cmix/releases
    Changes from version 17 to version 18:
    - LSTM improvements

    @Byron: if it is not a problem, could you announce in this thread when you release a new version of cmix? TIA

  24. Thanks:

    Darek (3rd August 2019)

  25. #409
    Member
    Join Date
    Mar 2011
    Location
    USA
    Posts
    232
    Thanks
    109
    Thanked 112 Times in 68 Posts
    Quote Originally Posted by Mauro Vezzosi View Post
    @Byron: if it is not a problem, could you announce in this thread when you release a new version of cmix? TIA
    Sure. Here are some results:

    Compressed size of enwik8: 14838332 bytes
    Compressed size of enwik9: 115714367 bytes


    Size of source code as a zip file: 208,961 bytes
    zip file contains:
    - all source code
    - makefile
    - dictionary


    enwik9 compression time: 602867.49 seconds
    enwik9 decompression time: 601569.89 seconds
    Approximate memory used: 25738196 KiB


    Here are the Silesia results for "precomp v0.4.7 -cn | cmix v18"


    dicke: 1813095
    mozil: 6717412
    mr: 1829883
    nci: 792994
    ooff: 1226244
    osdb: 1962336
    reym: 712062
    samba: 1614935
    sao: 3727061
    webst: 4297002
    x-ray: 3508509
    xml: 236101


    total: 28437634

  26. Thanks (3):

    Darek (4th August 2019),Mauro Vezzosi (3rd August 2019),Mike (4th August 2019)

  27. #410
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    962
    Thanks
    573
    Thanked 397 Times in 295 Posts
    Scores for my testset - 13KB of gain - nice, expecially due to "only" LSTM improvenments!
    Good scores for bigger and exe files.
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	cmix_v18.jpg 
Views:	33 
Size:	642.8 KB 
ID:	6754  

  28. #411
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    962
    Thanks
    573
    Thanked 397 Times in 295 Posts
    cmix v18 scores for 4 corpuses - mostly based on Byron's scores (thanks to post it on cmix page!).
    In general for bigger coruses there are significant gain. For smaller files (Calgery, Canterbury) not so big...
    One advantage over v17 version is a little sppedup (on my laptop) = compressing times of cmix v18 version are about 96% of cmix v17 - then there are good compression gain and a little sppedup in the same version!

    In the second table there are the best scores for the 4 Corpuses on August 2019
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	cmix_v18_4_Corpuses.jpg 
Views:	39 
Size:	1.51 MB 
ID:	6772   Click image for larger version. 

Name:	4_Corpuses_max_August_2019.jpg 
Views:	32 
Size:	591.3 KB 
ID:	6773  

  29. #412
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    75
    Thanks
    3
    Thanked 9 Times in 9 Posts
    Quote Originally Posted by Darek View Post
    cmix v18 scores for 4 corpuses - mostly based on Byron's scores (thanks to post it on cmix page!).
    In general for bigger coruses there are significant gain. For smaller files (Calgery, Canterbury) not so big...
    One advantage over v17 version is a little sppedup (on my laptop) = compressing times of cmix v18 version are about 96% of cmix v17 - then there are good compression gain and a little sppedup in the same version!

    In the second table there are the best scores for the 4 Corpuses on August 2019

    How to compile cmix on devcpp++ ? I am interested in data compression too beside computer vision. This is my little project on computer vision under Android.
    https://youtu.be/ofwEuCclswM

  30. Thanks:

    bwt (7th November 2019)

  31. #413
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,423
    Thanks
    223
    Thanked 1,053 Times in 565 Posts
    Just download http://www.byronknoll.com/cmix-v18.zip and compile normally?
    It compiles with
    find . -iname '*.cpp' >list
    g++ @list
    so just add all cpp files to a project or something?

  32. #414
    Member Gotty's Avatar
    Join Date
    Oct 2017
    Location
    Hungary
    Posts
    399
    Thanks
    278
    Thanked 282 Times in 149 Posts
    Quote Originally Posted by suryakandau@yahoo.co.id View Post
    How to compile cmix on devcpp++ ?
    devcpp is 4 years old.
    I haven't tried, but I suspect it won't compile cmix out-of-the-box.
    You should switch to a more recent development environment.

  33. #415
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,423
    Thanks
    223
    Thanked 1,053 Times in 565 Posts
    I tried and cmix18 compiles with gcc 5.10, but with "g++ -std=gnu++0x -static @list", there're errors without "-std"

  34. Thanks:

    Gotty (6th November 2019)

  35. #416
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    75
    Thanks
    3
    Thanked 9 Times in 9 Posts
    Quote Originally Posted by Shelwien View Post
    I tried and cmix18 compiles with gcc 5.10, but with "g++ -std=gnu++0x -static @list", there're errors without "-std"
    i am using win7 64bit. i have copied cmix folder to drive c: and then rename makefile to makefile.bat and then run makefile.bat but still don't produce executable file

  36. #417
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,423
    Thanks
    223
    Thanked 1,053 Times in 565 Posts
    makefile is not a shell script, different syntax.
    Unpack attached scripts to cmix\src\, update gcc path in g.bat (C:\MinGW510\bin to your path with g++.exe), then run g.bat.
    Attached Files Attached Files

  37. #418
    Member
    Join Date
    Sep 2015
    Location
    Italy
    Posts
    250
    Thanks
    109
    Thanked 146 Times in 107 Posts
    Cmix commit 2019/12/05, changing from layer_norm to rms_norm: yes, rms_norm looks better than layer_norm.
    How much does cmix (or lstm-compress) improve?

  38. #419
    Member
    Join Date
    Mar 2011
    Location
    USA
    Posts
    232
    Thanks
    109
    Thanked 112 Times in 68 Posts
    Quote Originally Posted by Mauro Vezzosi View Post
    Cmix commit 2019/12/05, changing from layer_norm to rms_norm: yes, rms_norm looks better than layer_norm.
    How much does cmix (or lstm-compress) improve?
    cmix on enwik8: 14838332 -> 14834133
    cmix on enwik9: 115714367 -> 115638939

  39. Thanks:

    Mauro Vezzosi (9th December 2019)

Page 14 of 14 FirstFirst ... 4121314

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •