Page 2 of 2 FirstFirst 12
Results 31 to 45 of 45

Thread: Integer compression

  1. #31
    Member
    Join Date
    Apr 2012
    Location
    London
    Posts
    265
    Thanks
    13
    Thanked 0 Times in 0 Posts
    Input 00 is A , 01 is B , 10 is C , 11 is D ( not 0 10 110 111 )

  2. #32
    Member Gotty's Avatar
    Join Date
    Oct 2017
    Location
    Switzerland
    Posts
    555
    Thanks
    356
    Thanked 359 Times in 195 Posts
    What you write does not make sense.

    Again:
    In your concrete case ("A is 0 B is 10 C is 110 D is 111") how many outputs do you have? How do you interpret them?

    Quote Originally Posted by LawCounsels View Post
    A(00) B(01) C(10) D(11) and all exact same probabilities ( 50% , 25%, 12.5% , 12.5% )
    Please note that for those probabilities the correct Huffman encoding is "A is 0 B is 10 C is 110 D is 111" as you wrote earlier, and not "A(00) B(01) C(10) D(11)".

    If you want to teach your neural network to Huffman encode a file (which is not possible) you'll need to stick to the rules.
    Please, first think, then answer.

  3. #33
    Member
    Join Date
    Apr 2012
    Location
    London
    Posts
    265
    Thanks
    13
    Thanked 0 Times in 0 Posts
    it's been consistent from beginning in input 1,000 bits '00' is A, 01 is B, 10 is C, 11 is D

    We compress with Huffman code table an A (00) in input produces 0 , an input B (01) produces 10.. so forth

  4. #34
    Member Gotty's Avatar
    Join Date
    Oct 2017
    Location
    Switzerland
    Posts
    555
    Thanks
    356
    Thanked 359 Times in 195 Posts
    I'm asking about your outputs.
    See?
    Quote Originally Posted by Gotty View Post
    In your concrete case ("A is 0 B is 10 C is 110 D is 111") how many outputs do you have? How do you interpret them?
    Quote Originally Posted by Gotty View Post
    In your concrete case ("A is 0 B is 10 C is 110 D is 111") how many outputs do you have? How do you interpret them?
    So you have "00 01 10 00 11 00 00 01" ("ABCADAAB") on your input (just 16 bits to simplify the case). You want to find the Huffman encoded format, which is "0 10 110 0 111 0 0 10" What is your expected output of the Neural Network?

  5. #35
    Member
    Join Date
    Apr 2012
    Location
    London
    Posts
    265
    Thanks
    13
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by Gotty View Post
    I'm asking about your outputs.
    See?



    So you have "00 01 10 00 11 00 00 01" ("ABCADAAB") on your input (just 16 bits to simplify the case). You want to find the Huffman encoded format, which is "0 10 110 0 111 0 0 10" What is your expected output of the Neural Network?
    with 16 bits input that will be 8 input symbols ( A B C D), which occurs 4x 2x 1x 1x. Thus output will always be log(base2)[C(8,4,2,1,1)] x 2 number of nodes ( each node corresponds to 1 output bit) ALWAYS

  6. #36
    Member
    Join Date
    Apr 2012
    Location
    London
    Posts
    265
    Thanks
    13
    Thanked 0 Times in 0 Posts
    or just ADD 4*1 + 2*2 + 1*3 + 1*3 = 14 output layer nodes ALWAYS

  7. #37
    Member
    Join Date
    Apr 2012
    Location
    London
    Posts
    265
    Thanks
    13
    Thanked 0 Times in 0 Posts
    These 14 output bits consists of unique prefixes 0 10 110 or 111, unambiguous interpretable sequential

  8. #38
    Member Gotty's Avatar
    Join Date
    Oct 2017
    Location
    Switzerland
    Posts
    555
    Thanks
    356
    Thanked 359 Times in 195 Posts
    We agree on that. See my post. I have 14 bits.
    So you will have 14 output nodes giving you some values between 0 and 1. I suppose you will interpret an output as 1 when it is over 0.5 and 0 when it is less. Right?
    You train your (general feed-forward neural network) with some cost function. The training needs to be 100% perfect. If it does not produce the expected result even in one of the cases, you are in trouble. If your training set does not contain all the strings you have a risk that the network would not produce the correct, expected result for the missing cases.
    So your only chance is to feed all possible input strings to the neural network and train the network to produce 100% results on the output. It can be done with 16 input bits, yes.
    Now let's go back to 1000 input bits (1024 to fulfill input requirements). Feed all possible 1024 bit stings to the inputs... Oh... how much time do you have?

  9. #39
    Member
    Join Date
    Apr 2012
    Location
    London
    Posts
    265
    Thanks
    13
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by Gotty View Post
    We agree on that.
    Now let's go back to 1000 input bits (1024 to fulfill input requirements). Feed all possible 1024 bit stings to the inputs... Oh... how much time do you have?
    This is trivial you can just initialize with correct crafted sets of weights/bias AND immediate correct for ALL 2^1000 possible inputs SINCE weights here really just exactly do what algorithm does ( 100% correct ALL the Times)

    No training required even in this trivial case

  10. #40
    Member Gotty's Avatar
    Join Date
    Oct 2017
    Location
    Switzerland
    Posts
    555
    Thanks
    356
    Thanked 359 Times in 195 Posts
    I'm not sure if it's that trivial. Can you provide the weights? Please try it. For the 16-bit case.

    Anyway, if you precalculate the weights your whole point becomes invalid. Which is:
    Quote Originally Posted by LawCounsels View Post
    relaxes the requirements now allowed to train feed forward NN with target optimum Huffman encoded compressed of training input sets , can it now learn to correct Huffman encode tests new input sets ( not even requires that it now discovered Huffman code ) ?
    Quote Originally Posted by LawCounsels View Post
    ie if we now train the feed forward NN with training input binaries data AND comparing to known Huffman encoded compressed output binaries data to back-propagate adjust weights/ bias , can the NN then correct Huffman encode entire new test input binaries data produces correct compressed output binaries henceforth ?
    (Boldface = my emphasis)

  11. #41
    Member
    Join Date
    Apr 2012
    Location
    London
    Posts
    265
    Thanks
    13
    Thanked 0 Times in 0 Posts
    Yes to be general applicable ( eventual direction) we should just fix some sets of weights so that successive 2 input bits 'hardwired' to produce sequentially 0 10 110 111 ( later eventual direction for training to adapt accommodate various encodings, output layers can now be extra provision ie 3 bits * 500 nodes... 1st '0' delimites Eg 0 or 10 or 110 OR all 111 delimiter. Can generalise this even further later)

    Just needs 'hardwired' certain weights so that successive 2 input bits produces sequential codewords ( onto over-provisioned N bits output nodes each)

  12. #42
    Member
    Join Date
    Apr 2012
    Location
    London
    Posts
    265
    Thanks
    13
    Thanked 0 Times in 0 Posts
    Concatenate ALL 500 delimited strings 0 or 10 or 110 or 111 from successive 3 output nodes GIVES Huffman encoded compression

  13. #43
    Member Gotty's Avatar
    Join Date
    Oct 2017
    Location
    Switzerland
    Posts
    555
    Thanks
    356
    Thanked 359 Times in 195 Posts
    It looks like finding the weights is far from 'trivial'. If you do think it's trivial you will easily post the weights of the 16-bit case.
    Nevertheless your question was about neural network training and learning. I hope you already see that it's not really possible to train a neural network to perfectly emulate Huffman encoding.

  14. #44
    Member
    Join Date
    Apr 2012
    Location
    London
    Posts
    265
    Thanks
    13
    Thanked 0 Times in 0 Posts
    I am occupied elsewhere for sometime

    Let's see if some practitioners already know it's obvious 'perfect' trained with the 'hardwired'

  15. #45
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,572
    Thanks
    783
    Thanked 687 Times in 372 Posts
    Of course, once you walked in with genius idea of using NN for XYZ, the rest of task is trivial and can be handled by any experts in XYZ hanging around. We just need to know your name to give you priority in saying that NN can be used for XYZ (jn this case, it's Huffman coding).

Page 2 of 2 FirstFirst 12

Similar Threads

  1. TurboPFor: Integer Compression
    By dnd in forum Data Compression
    Replies: 50
    Last Post: 15th November 2019, 15:48
  2. Replies: 4
    Last Post: 17th November 2016, 04:12
  3. Integer compression
    By Senoble in forum Data Compression
    Replies: 23
    Last Post: 24th November 2015, 12:05
  4. fastest open source integer compression algorithms
    By mitra in forum Data Compression
    Replies: 2
    Last Post: 5th September 2015, 18:21
  5. Replies: 4
    Last Post: 22nd June 2015, 00:32

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •