Input 00 is A , 01 is B , 10 is C , 11 is D ( not 0 10 110 111 )
Input 00 is A , 01 is B , 10 is C , 11 is D ( not 0 10 110 111 )
What you write does not make sense.
Again:
In your concrete case ("A is 0 B is 10 C is 110 D is 111") how many outputs do you have? How do you interpret them?
Please note that for those probabilities the correct Huffman encoding is "A is 0 B is 10 C is 110 D is 111" as you wrote earlier, and not "A(00) B(01) C(10) D(11)".
If you want to teach your neural network to Huffman encode a file (which is not possible) you'll need to stick to the rules.
Please, first think, then answer.
it's been consistent from beginning in input 1,000 bits '00' is A, 01 is B, 10 is C, 11 is D
We compress with Huffman code table an A (00) in input produces 0 , an input B (01) produces 10.. so forth
I'm asking about your outputs.
See?
So you have "00 01 10 00 11 00 00 01" ("ABCADAAB") on your input (just 16 bits to simplify the case). You want to find the Huffman encoded format, which is "0 10 110 0 111 0 0 10" What is your expected output of the Neural Network?
or just ADD 4*1 + 2*2 + 1*3 + 1*3 = 14 output layer nodes ALWAYS
These 14 output bits consists of unique prefixes 0 10 110 or 111, unambiguous interpretable sequential
We agree on that. See my post. I have 14 bits.
So you will have 14 output nodes giving you some values between 0 and 1. I suppose you will interpret an output as 1 when it is over 0.5 and 0 when it is less. Right?
You train your (general feed-forward neural network) with some cost function. The training needs to be 100% perfect. If it does not produce the expected result even in one of the cases, you are in trouble. If your training set does not contain all the strings you have a risk that the network would not produce the correct, expected result for the missing cases.
So your only chance is to feed all possible input strings to the neural network and train the network to produce 100% results on the output. It can be done with 16 input bits, yes.
Now let's go back to 1000 input bits (1024 to fulfill input requirements). Feed all possible 1024 bit stings to the inputs... Oh... how much time do you have?
This is trivial you can just initialize with correct crafted sets of weights/bias AND immediate correct for ALL 2^1000 possible inputs SINCE weights here really just exactly do what algorithm does ( 100% correct ALL the Times)
No training required even in this trivial case
Yes to be general applicable ( eventual direction) we should just fix some sets of weights so that successive 2 input bits 'hardwired' to produce sequentially 0 10 110 111 ( later eventual direction for training to adapt accommodate various encodings, output layers can now be extra provision ie 3 bits * 500 nodes... 1st '0' delimites Eg 0 or 10 or 110 OR all 111 delimiter. Can generalise this even further later)
Just needs 'hardwired' certain weights so that successive 2 input bits produces sequential codewords ( onto over-provisioned N bits output nodes each)
Concatenate ALL 500 delimited strings 0 or 10 or 110 or 111 from successive 3 output nodes GIVES Huffman encoded compression
It looks like finding the weights is far from 'trivial'. If you do think it's trivial you will easily post the weights of the 16-bit case.
Nevertheless your question was about neural network training and learning. I hope you already see that it's not really possible to train a neural network to perfectly emulate Huffman encoding.
I am occupied elsewhere for sometime
Let's see if some practitioners already know it's obvious 'perfect' trained with the 'hardwired'
Of course, once you walked in with genius idea of using NN for XYZ, the rest of task is trivial and can be handled by any experts in XYZ hanging around. We just need to know your name to give you priority in saying that NN can be used for XYZ (jn this case, it's Huffman coding).