Page 3 of 5 FirstFirst 12345 LastLast
Results 61 to 90 of 138

Thread: NNCP: Lossless Data Compression with Neural Networks

  1. #61
    Member
    Join Date
    Mar 2011
    Location
    USA
    Posts
    252
    Thanks
    116
    Thanked 124 Times in 72 Posts
    @Fabrice, thanks for your answers so far in this forum, they are useful. I have been working on adding layer norm to cmix. So far it does slightly worse than the original, but I am still tuning parameters. I am also hoping to experiment soon with using batching to speed up the LSTM.

    I posted a question to StackExchange here. cmix/lstm-compress use technique#2 from that question. Does NNCP also use technique#2? Did you experiment with other techniques?

    For adding batching, it seems like I can just run backpropagation every (
    batch_size * time_steps) symbols. It also seems like I can just pass the LSTM state forward normally. I am a bit confused about how NNCP does batching - I don't see why it is necessary to split and compress independently. Did you experiment with different ways to do batches? I'm not sure if what I have in mind for batching makes sense - maybe I can post a diagram of my idea if it is useful.
    Last edited by byronknoll; 29th May 2019 at 20:08.

  2. #62
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,269
    Thanks
    801
    Thanked 544 Times in 414 Posts
    Quote Originally Posted by fab View Post
    Did you tune the learning rate (-lr option) ? When you change the model parameters (such as the batch_size), it is important to retune the learning rate.

    For example, with hidden_size = 96, n_layer=3, fc=1, batch_size=16, the best learning rate is 0.01. When reducing batch_size to 1, a better learning rate is 0.0032.
    Fabrice, the parameters wich I've already tested (or there are in progress) are:
    1) methods: LSTM vs. LSTM-C vs. LSTM-T vs. LSTM LARGE
    2) options: FULL CONNECT =1/0, NORMALISATION =1/0 and combinations of it
    3) BATCH SIZE 1,2,3,4,5,6,7,8,9,10,11,12,14,16,20,24,28,32
    4) N LAYERS = 1..10
    5) HIDDEN SIZE = 16,32,64,128,160,192,224,256,288,320,352,384,416,4 48,512,576,640,768,1024 (in progress)

    And then I've plan to test LEARNIG RATE option at the end. I didn't set the test scope for this option - do you suggest something?
    Of course I understand that to be ideallky proper then for every option I should test all options but it's impossible then my best settings would be suboptimal.

    It takes time. From strange reason for most of my actual options (and RC_v1 version) top speed is for 1 thread which is a change from default option when 4 threads was the fastests.
    It's theoretically possible to run 4-6 instances which are parallelly in total faster than one instance at the time but from personal reasons I cannot run now more instances due to very loud work of my laptop... , however maybe later I'll run in that way. But I'm very curious and I'l find to find best possible settings for NNCP

  3. #63
    Member
    Join Date
    Sep 2015
    Location
    Italy
    Posts
    287
    Thanks
    120
    Thanked 166 Times in 123 Posts
    Quote Originally Posted by Darek View Post
    Is it lsym-compress from (5) your modified version?
    Yes, if you (and Byron) want I'll try to parameterize some options and post source+exe.
    I finished (5).

    @Fabrice
    Thanks, now it's clearer, I've also paid more attention to "batch_size" in the source code.
    For you "batch_size" is the number of streams simultaneously compressed with the same NN, for me "the batch size is [...] the number of samples to work through before updating the internal model parameters" (= "batch_size * time_steps" symbols, as you wrote).

  4. Thanks:

    Darek (30th May 2019)

  5. #64
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,269
    Thanks
    801
    Thanked 544 Times in 414 Posts
    Quote Originally Posted by Mauro Vezzosi View Post
    Yes, if you (and Byron) want I'll try to parameterize some options and post source+exe.
    I finished (5).
    Yes, If you would like to.

    Looks that your optimizations gives about 1MB of gain! That's 6% of improvements!

  6. #65
    Member
    Join Date
    Apr 2019
    Location
    France
    Posts
    20
    Thanks
    0
    Thanked 35 Times in 17 Posts
    Quote Originally Posted by byronknoll View Post
    @Fabrice, thanks for your answers so far in this forum, they are useful. I have been working on adding layer norm to cmix. So far it does slightly worse than the original, but I am still tuning parameters. I am also hoping to experiment soon with using batching to speed up the LSTM.

    I posted a question to StackExchange here. cmix/lstm-compress use technique#2 from that question. Does NNCP also use technique#2? Did you experiment with other techniques?

    For adding batching, it seems like I can just run backpropagation every (
    batch_size * time_steps) symbols. It also seems like I can just pass the LSTM state forward normally. I am a bit confused about how NNCP does batching - I don't see why it is necessary to split and compress independently. Did you experiment with different ways to do batches? I'm not sure if what I have in mind for batching makes sense - maybe I can post a diagram of my idea if it is useful.
    I confirm NNCP uses technique #2. I did not experiment with other techniques. For example, it may be interesting to test a backpropagation every time_steps / 2 instead of every time_steps (overlap of time_steps / 2 symbols). There are also several possibilities regarding the contribution of the overlapped symbols to the loss function.

    I don't see another way to implement batches (they are just independent streams of symbols used in a single backpropagation step). However, there may be models giving a higher parallelism with a smaller batch size. For example the Transformer model gives more parallelism than LSTM, but unfortunately it is only true for the encoding side.

  7. Thanks:

    byronknoll (20th June 2019)

  8. #66
    Member
    Join Date
    Apr 2019
    Location
    France
    Posts
    20
    Thanks
    0
    Thanked 35 Times in 17 Posts
    Quote Originally Posted by Darek View Post
    Fabrice, the parameters wich I've already tested (or there are in progress) are:
    1) methods: LSTM vs. LSTM-C vs. LSTM-T vs. LSTM LARGE
    2) options: FULL CONNECT =1/0, NORMALISATION =1/0 and combinations of it
    3) BATCH SIZE 1,2,3,4,5,6,7,8,9,10,11,12,14,16,20,24,28,32
    4) N LAYERS = 1..10
    5) HIDDEN SIZE = 16,32,64,128,160,192,224,256,288,320,352,384,416,4 48,512,576,640,768,1024 (in progress)

    And then I've plan to test LEARNIG RATE option at the end. I didn't set the test scope for this option - do you suggest something?
    Of course I understand that to be ideallky proper then for every option I should test all options but it's impossible then my best settings would be suboptimal.

    It takes time. From strange reason for most of my actual options (and RC_v1 version) top speed is for 1 thread which is a change from default option when 4 threads was the fastests.
    It's theoretically possible to run 4-6 instances which are parallelly in total faster than one instance at the time but from personal reasons I cannot run now more instances due to very loud work of my laptop... , however maybe later I'll run in that way. But I'm very curious and I'l find to find best possible settings for NNCP
    You can use a set of learning rates increasing geometrically and aligned on powers of 10, such as 10^(n/4) where n = -16 ... -8.

  9. Thanks:

    Darek (20th June 2019)

  10. #67
    Member
    Join Date
    Mar 2011
    Location
    USA
    Posts
    252
    Thanks
    116
    Thanked 124 Times in 72 Posts
    I just noticed Fabrice released a new compression tool here: http://textsynth.org/sms.html

    It uses the 345M version of GPT-2 (which acheives better compression rates for language modelling than cmix if you don't take into account model size).

  11. Thanks (4):

    Cyan (20th June 2019),Mauro Vezzosi (19th June 2019),Shelwien (19th June 2019),xinix (20th June 2019)

  12. #68
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,269
    Thanks
    801
    Thanked 544 Times in 414 Posts
    Here are scores of my testset for HIDDEN SIZE parameters. It takes some time due to very slow compression at the settings above 512 for this parameter.
    However some files got improvements from 1024 settings and maybe higher.
    Then I'll test learning rate option.
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	nncp_hidden_size.jpg 
Views:	106 
Size:	1.70 MB 
ID:	6670  

  13. #69
    Member
    Join Date
    Apr 2019
    Location
    France
    Posts
    20
    Thanks
    0
    Thanked 35 Times in 17 Posts
    A new version of NNCP is available at https://bellard.org/nncp . The overhead of the arithmetic coder was reduced and the code is slightly faster when using threads. I tested a larger model (see readme.txt) which gives better results on enwik8/enwik9 (decompression still in progress):

    enwik8: 16 571 476 bytes
    enwik9: 123 050 014 bytes

  14. Thanks (4):

    byronknoll (30th June 2019),Darek (29th June 2019),Jyrki Alakuijala (30th June 2019),Mauro Vezzosi (29th June 2019)

  15. #70
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    976
    Thanks
    266
    Thanked 350 Times in 221 Posts
    This is of course interesting, but is it philosophically different from a version of brotli with 7000x larger static dictionary possibly with some adjustments to transforms to accommodate a larger model more efficiently.

    People seemed to be worried about the static dictionary (~50 kB compressed) that brotli brings along, but heads are nodding for a 345 MB static dictionary for neural modeling.

  16. #71
    Member
    Join Date
    Sep 2015
    Location
    Italy
    Posts
    287
    Thanks
    120
    Thanked 166 Times in 123 Posts
    @Jyrki: it seems to me that in machine learning people consider it more important to take into account the number of parameters rather than the time or memory used.
    nncp does not use the 345M version of GPT-2, the larger model of the 2019/06/29 version uses hidden_size = 512 instead of the previous 384.
    From readme.txt:
    2019/05/08 ./nncp -n_layer 7 -hidden_size 384 -n_embed_out 5 -n_symb 16388 -full_connect 1 -lr 6e-3 c out.pre out.bin
    2019/06/29 ./nncp -n_layer 7 -hidden_size 512 -n_embed_out 5 -n_symb 16388 -full_connect 1 -lr 5e-3 -block_len 500000 c out.pre out.bin

    @Byron: did you see that nncp uses log softmax instead of softmax? (nncp.c 2019/06/29, line 382)
    Some links:
    https://datascience.stackexchange.co...-softmax/40719
    https://discuss.pytorch.org/t/what-i...-softmax/11801
    https://github.com/deeplearning4j/nd4j/issues/2822

  17. Thanks:

    byronknoll (30th June 2019)

  18. #72
    Member
    Join Date
    Mar 2011
    Location
    USA
    Posts
    252
    Thanks
    116
    Thanked 124 Times in 72 Posts
    @Jyrki, also keep in mind that the neural networks in nncp and cmix are trained from scratch when compressing files, so it isn't necessary to save or transfer the model.

  19. #73
    Member
    Join Date
    Mar 2011
    Location
    USA
    Posts
    252
    Thanks
    116
    Thanked 124 Times in 72 Posts
    Quote Originally Posted by Mauro Vezzosi View Post
    @Byron: did you see that nncp uses log softmax instead of softmax? (nncp.c 2019/06/29, line 382)
    I don't understand how to implement this. Does this change the objective function during backprop, or is cross entropy (aka log loss) still used? If the objective function is the same, it isn't really important if the final probability is stored with a log or not.

  20. #74
    Member
    Join Date
    Sep 2015
    Location
    Italy
    Posts
    287
    Thanks
    120
    Thanked 166 Times in 123 Posts
    I still don't know much about log softmax, I saw that nccp uses it, I assumed it had advantages over softmax and worth reporting it.

  21. #75
    Member
    Join Date
    Apr 2019
    Location
    France
    Posts
    20
    Thanks
    0
    Thanked 35 Times in 17 Posts
    Quote Originally Posted by Mauro Vezzosi View Post
    I still don't know much about log softmax, I saw that nccp uses it, I assumed it had advantages over softmax and worth reporting it.
    NNCP uses the standard softmax as lstm-compress. nc_log_soft_max() computes the softmax but also computes log(p_i) where p_i is the selected element of the softmax in order to get the loss function.

  22. Thanks:

    Mauro Vezzosi (5th July 2019)

  23. #76
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    976
    Thanks
    266
    Thanked 350 Times in 221 Posts
    Quote Originally Posted by byronknoll View Post
    @Jyrki, also keep in mind that the neural networks in nncp and cmix are trained from scratch when compressing files, so it isn't necessary to save or transfer the model.
    Cool! Is it practical to do learning at compression time? What is the current speed record?

  24. #77
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    4,133
    Thanks
    320
    Thanked 1,396 Times in 801 Posts
    nncp is like paq or cmix, so speed record is 80 bytes per second or something... normally it does around 2kb/s with MT.

    The point is that compression gives an objective metric of prediction quality, and thus NN library quality.

  25. #78
    Member
    Join Date
    Feb 2016
    Location
    U.S.
    Posts
    5
    Thanks
    7
    Thanked 0 Times in 0 Posts
    I'm having trouble opening nncp. I tried downloading nncp-2019-06-29.tar.gz from the website. When I try extracting it on Ubuntu 19.04, I get an error message:
    Code:
    gzip: stdin: not in gzip formattar: Child returned status 1
    tar: Error is not recoverable: exiting now



  26. #79
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    4,133
    Thanks
    320
    Thanked 1,396 Times in 801 Posts
    Its actually simply tar, without gzip, "tar -xf nncp-2019-06-29.tar.gz" should extract it.

  27. Thanks:

    137ben (11th August 2019)

  28. #80
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,269
    Thanks
    801
    Thanked 544 Times in 414 Posts
    Scores of my testset for NNCP learning rate setups from 0.1 to 0.0001.
    One trick - for files which have most demanding hidden size setting (1024), I've tested setting = 640 due to time boundaries, but I'll test best ln options with this hs setting.

    The best score is for 0.0008 setting (default is 0.004). Most best scores are for 0.002 setting.
    The biggest file (K.WAD) is for 0.0003 setting, however other big or most learning rate affected files have their minimum about 0.0009 - 0.0008 setting.

    To sum up - as fab wrote - learning rate optimization got most spectacular improvements.
    In summary of all optimizations -> i"ve got 6.9% (1MB) of improvement to default option which is a very good improvement rate!
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	nncp_learning_rate.jpg 
Views:	66 
Size:	2.17 MB 
ID:	6788  

  29. Thanks:

    Mauro Vezzosi (13th August 2019)

  30. #81
    Member
    Join Date
    Sep 2015
    Location
    Italy
    Posts
    287
    Thanks
    120
    Thanked 166 Times in 123 Posts
    Tested -time_steps and -seed on the following 3 files to check if and how much they gain.
    1.BMP
    333.003 (a) = Darek's min nncp
    332.617 (b) = (a) + -time_steps 17
    331.765 (c) = (b) + -hidden_size 192
    331.746 (d) = (c) + -adam_beta2 0.999
    No gain was found by changing -seed.
    O.APR
    6.052 (a) = Darek's min nncp
    6.045 (b) = (a) + -seed 0
    No gain was found by changing -time_steps.
    R.DOC
    34.802 (a) = Darek's min nncp
    34.692 (b) = (a) + -time_steps 17
    34.659 (c) = (b) + -seed 1

  31. Thanks:

    Darek (16th August 2019)

  32. #82
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,269
    Thanks
    801
    Thanked 544 Times in 414 Posts
    @Mauro - looks promising . Did you implement it in new NNCP version?
    Update the scores for K.WAD and L.PAK - I've tested these files with hidden size 1024 settings and got another 45KB of gain.
    In total my optimization gives 7.2% compared to default option (almost 1.1MB of gain). In this way NNCP got 13'th place from all compressors - w/o any models it's impressive!
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	Ranking_Total.jpg 
Views:	82 
Size:	417.8 KB 
ID:	6800  

  33. #83
    Member
    Join Date
    Sep 2015
    Location
    Italy
    Posts
    287
    Thanks
    120
    Thanked 166 Times in 123 Posts
    Quote Originally Posted by Darek View Post
    @Mauro - Did you implement it in new NNCP version?
    No, they exist since the first version of NNCP.

  34. #84
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,269
    Thanks
    801
    Thanked 544 Times in 414 Posts
    Ok, then I have some parameters to test.

  35. #85
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,269
    Thanks
    801
    Thanked 544 Times in 414 Posts
    @Mauro - did you have parameters range whioch can be used for time_steps, adam_beta2 and seed ?

  36. #86
    Member
    Join Date
    Sep 2015
    Location
    Italy
    Posts
    287
    Thanks
    120
    Thanked 166 Times in 123 Posts
    Quote Originally Posted by Darek View Post
    @Mauro - did you have parameters range whioch can be used for time_steps, adam_beta2 and seed ?
    My opinion is:
    Code:
    Option           Description                            Default      Note
    -time_steps n    Number of time steps for TBTT          20           No special suggestion, say 5-30. Test 18, 22, 16, 24, ... and go in the best direction.
    -seed n          Random number generator seed           123          In some cases, initializing weights with different values slightly improves the compression (see "change seed from 1 to 2" in https://encode.su/threads/2882-lstm-...ll=1#post61172).
    -adam_beta1 n    ADAM beta1 parameter                   0.0          The widely suggested value to set is 0.9, I tested 0.01 and 0.1, maybe it can also be tested 0.001 (and, of course, any other values < 1.0).
    -adam_beta2 n    ADAM beta2 parameter                   0.9999       The widely suggested value to set is 0.999, I tested 0.999, maybe it can also be tested 0.99999 (and, of course, any other values < 1.0).
    -adam_eps n      ADAM epsilon parameter                 0.00001      I tested 0.000001, 0.0000001, 0.00000001 (any other value, say < 0.0001, can be tested). Must be > 0.
    -n_embed_out n   Number of layers in output embedding   = -n_layer   Try n_layer - 1, n_layer - 2, ... (expecially when n_layer is "big"). Must be >= 1.

  37. Thanks:

    Darek (23rd August 2019)

  38. #87
    Member
    Join Date
    Oct 2010
    Location
    Germany
    Posts
    289
    Thanks
    10
    Thanked 34 Times in 22 Posts
    Why don't you guys use an automatic optimization tool instead of trying out parameters by hand. It's not that these are independent from another.

  39. #88
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,269
    Thanks
    801
    Thanked 544 Times in 414 Posts
    @Sebastian - about what kind of autmatic optimization tool do you wrote? I've must miss something...
    I've used hands parameters settings to review full scoope of parameters, which is more time consuming and not optimal - that's absolutely right - then auto-optimization tool would be probably more accurate.

  40. #89
    Member
    Join Date
    Oct 2010
    Location
    Germany
    Posts
    289
    Thanks
    10
    Thanked 34 Times in 22 Posts
    Quote Originally Posted by Darek View Post
    @Sebastian - about what kind of autmatic optimization tool do you wrote? I've must miss something...
    I've used hands parameters settings to review full scoope of parameters, which is more time consuming and not optimal - that's absolutely right - then auto-optimization tool would be probably more accurate.
    I have written one. It works on config files und uses stochastic search methods. Even multithreading is possible. To make it use with command line compressors we can add a simple python script to the pipeline. I'll take a look at it next week.

  41. Thanks:

    Darek (23rd August 2019)

  42. #90
    Member
    Join Date
    Sep 2015
    Location
    Italy
    Posts
    287
    Thanks
    120
    Thanked 166 Times in 123 Posts
    Some tests of nncp and trfcp.
    Code:
    nncp 1 : release 2019/04/13 and 2019/05/08  , default option (cell=LSTM-C n_layer=4 hidden_size=352 batch_size=16 time_steps=20 n_symb=256 ln=1 fc=0 sgd_opt=adam lr=4.000e-003 beta1=0.000000 beta2=0.999900 eps=1.000e-005 n_params=5.28M n_params_nie=3.84M mem=79.1MB (Windows Task Manager reports 85900- 88220 K)).
    nncp 2 : release Shelwien's rc v1 2019/04/28, default option (cell=LSTM-C n_layer=4 hidden_size=352 batch_size=16 time_steps=20 n_symb=256 ln=1 fc=0 sgd_opt=adam lr=4.000e-003 beta1=0.000000 beta2=0.999900 eps=1.000e-005 n_params=5.28M n_params_nie=3.84M mem=79.1MB (Windows Task Manager reports 99496-101455 K)).
    nncp 3 : release 2019/06/29                 , default option (cell=LSTM-C n_layer=4 hidden_size=352 batch_size=16 time_steps=20 n_symb=256 ln=1 fc=0 sgd_opt=adam lr=4.000e-003 beta1=0.000000 beta2=0.999900 eps=1.000e-005 n_params=5.28M n_params_nie=3.84M mem=82.7MB (Windows Task Manager reports 89648- 92132 K)).
    trfcp 1: release 2019/06/29                 , default option (            n_layer=4 d_model=256 n_head=8 mem_len=64 train_len=64 d_pos=16 d_inner=512 n_symb=256 tied=0 init_range=0.050000 lr=3.00e-004,5000000,1.00e-004   n_params=2.23M n_params_nie=2.17M mem=26.3MB (Windows Task Manager reports 27108- 27248 K)).
    
        nncp 1     nncp 2     nncp 3    trfcp 1 Maximum Compression
       818.840    815.605    818.814    819.676 A10.jpg
     1.189.590  1.169.460  1.183.244  1.262.334 AcroRd32.exe
       465.370    454.447    451.084    447.945 english.dic
     3.671.152  3.649.121  3.667.985  3.679.094 FlashMX.pdf
       653.690    527.601    547.467    423.253 FP.LOG
     1.496.712  1.479.836  1.492.574  1.548.619 MSO97.DLL
       783.988    757.544    769.251    836.909 ohs.doc
       706.281    697.511    692.616    699.139 rafale.bmp
       547.304    530.862    534.520    613.083 vcfiu.hlp
       485.039    477.996    475.882    554.648 world95.txt
    10.817.966 10.559.983 10.633.437 10.884.700 Total
    
        nncp 1     nncp 2     nncp 3    trfcp 1 LTCB
                       48        108         66 ENWIK0
                       93        117         76 ENWIK1
           236        213        220        169 ENWIK2
           829        792        816        753 ENWIK3
         4.795      4.702      4.780      5.402 ENWIK4
        34.264     33.761     34.164     42.312 ENWIK5
       260.738    255.748    259.037    289.002 ENWIK6
     2.231.375  2.193.881  2.208.002  2.370.364 ENWIK7
    
    -------------------------------------------------------------
    
    First ~10 MB of ENWIK9 preprocessed as written in readme.txt:
    preprocess c out.words enwik9 out.pre 16384 512
    
    nncp 2 : release Shelwien's rc v1 2019/04/28, option (LSTM        large2 model): cell=LSTM-C n_layer=7 hidden_size=384 batch_size=16 time_steps=20 n_symb=16388 ln=1 fc=1               sgd_opt=adam lr=6.000e-003 beta1=0.000000 beta2=0.999900 eps=1.000e-005     n_params= 236M n_params_nie=60.6M mem=2.19GB (Windows Task Manager reports 2.174.224 K).
    nncp 3 : release 2019/06/29                 , option (LSTM        large2 model): cell=LSTM-C n_layer=7 hidden_size=384 batch_size=16 time_steps=20 n_symb=16388 ln=1 fc=1 n_embed_out=5 sgd_opt=adam lr=6.000e-003 beta1=0.000000 beta2=0.999900 eps=1.000e-005     n_params= 224M n_params_nie=48.0M mem=2.08GB (Windows Task Manager reports 2.056.248 K).
    trfcp 1: release 2019/06/29                 , option (Transformer large  model):             n_layer=6 d_model=512 n_head=8 mem_len=64 train_len=64 d_pos=64 d_inner=2048 n_symb=16388 tied=0 init_range=0.050000 lr=1.00e-004,5000000,5.00e-005,28000000,3.00e-005 n_params=35.7M n_params_nie=27.3M mem= 356MB (Windows Task Manager reports   351.032 K).
    
    |-------------------------------------nncp 2 |-------------------------------------nncp 3 |------------------------------------trfcp 1
        SIZE locBPS BPS/5M    BPS kS/s        LR     SIZE locBPS BPS/5M    BPS kB/s        LR     SIZE locBPS BPS/5M    BPS kS/s        LR
      100176 10.160 10.160 10.160 0.13 6.00e-003   100176 10.341 10.341 10.341 0.19 6.00e-003    99969 10.836 10.836 10.836 0.24 1.00e-004
      200336  8.978  9.569  9.569 0.13 6.00e-003   200336  9.012  9.676  9.676 0.20 6.00e-003   199938  9.482 10.159 10.159 0.24 9.90e-005
      300496  8.426  9.188  9.188 0.13 6.00e-003   300496  8.430  9.261  9.261 0.19 6.00e-003   299907  9.425  9.914  9.914 0.24 9.80e-005
      400656  7.972  8.884  8.884 0.13 6.00e-003   400656  7.983  8.942  8.942 0.20 6.00e-003   399876  9.042  9.696  9.696 0.24 9.70e-005
      500816  7.806  8.668  8.668 0.13 6.00e-003   500816  7.816  8.717  8.717 0.20 6.00e-003   499845  8.801  9.517  9.517 0.24 9.60e-005
      600976  7.838  8.530  8.530 0.13 6.00e-003   600976  7.844  8.571  8.571 0.19 6.00e-003   599814  8.759  9.391  9.391 0.24 9.50e-005
      701136  7.658  8.405  8.405 0.13 6.00e-003   701136  7.660  8.441  8.441 0.20 6.00e-003   699783  8.612  9.280  9.280 0.24 9.40e-005
      801296  7.490  8.291  8.291 0.13 6.00e-003   801296  7.492  8.322  8.322 0.20 6.00e-003   799752  8.047  9.125  9.125 0.24 9.30e-005
      901456  7.167  8.166  8.166 0.13 6.00e-003   901456  7.176  8.195  8.195 0.19 6.00e-003   899721  8.003  9.001  9.001 0.24 9.20e-005
     1001616  7.031  8.053  8.053 0.13 6.00e-003  1001616  7.046  8.080  8.080 0.19 6.00e-003   999690  7.967  8.897  8.897 0.24 9.10e-005
           ...                   ...          ...
     2003232  6.768  7.443  7.443 0.12 6.00e-003  2003232  6.755  7.454  7.454 0.20 6.00e-003  1999380  7.211  8.252  8.252 0.24 8.10e-005
     3004832  6.360  7.151  7.151 0.12 6.00e-003  3004832  6.351  7.155  7.155 0.19 6.00e-003  2999070  7.201  7.889  7.889 0.24 7.10e-005
     4006448  6.514  6.971  6.971 0.13 6.00e-003  4006448  6.510  6.972  6.972 0.20 6.00e-003  3998760  7.296  7.699  7.699 0.23 6.10e-005
     5008064  6.488  6.831  6.831 0.00 6.00e-003  5008064  6.476  6.828  6.828 0.20 6.00e-003  4998450  6.830  7.510  7.510 0.24 5.10e-005
     6009664  6.172  6.482  6.744 0.13 6.00e-003  6009664  6.160  6.473  6.741 0.18 6.00e-003  5998140  6.848  7.096  7.396 0.24 4.92e-005
     7011280  5.939  6.329  6.647 0.13 6.00e-003  7011280  5.922  6.318  6.643 0.20 6.00e-003  6997830  6.666  6.921  7.301 0.24 4.83e-005
     8012896  6.036  6.206  6.560 0.13 6.00e-003  8012896  6.023  6.196  6.555 0.17 6.00e-003  7997520  6.518  6.768  7.188 0.24 4.75e-005
     9014496  5.726  6.110  6.493 0.13 6.00e-003  9014496  5.725  6.099  6.487 0.20 6.00e-003  8997210  6.630  6.628  7.104 0.24 4.66e-005
    10016112  6.074  6.055  6.443 0.13 6.00e-003 10016112  6.065  6.045  6.437 0.20 6.00e-003  9996900  6.540  6.563  7.036 0.24 4.57e-005
                                                                                              10096869  6.701  6.556  7.033 0.24 4.57e-005
    Last edited by Mauro Vezzosi; 29th September 2019 at 21:08. Reason: Swapped the results of nncp 2 and nncp 3 in "First ~10 MB of ENWIK9 preprocessed".

Page 3 of 5 FirstFirst 12345 LastLast

Similar Threads

  1. Compression with recurrent neural networks
    By Matt Mahoney in forum Data Compression
    Replies: 6
    Last Post: 28th January 2019, 23:44
  2. The future of lossless data compression
    By inikep in forum Data Compression
    Replies: 66
    Last Post: 5th March 2018, 13:03
  3. Draco 3D data compression (lossless)
    By pothos2 in forum Data Compression
    Replies: 2
    Last Post: 27th January 2017, 01:27
  4. Replies: 5
    Last Post: 17th September 2015, 21:43
  5. lossless data compression
    By SLS in forum Data Compression
    Replies: 21
    Last Post: 15th March 2011, 12:35

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •