How Neural Network self-evolve?
Fundamentally, we can divide algorithms into two categories.
- One is we hard code an algorithm and get the desired results
- Second we design an intelligent algorithm which evaluate the latest information available and decide on the computational equation to be used to reach the final result
Let us suppose we want to distinguish between cats and dogs in the pictures given. In the first category we would give specifications like how a cat or a dog looks like. For example we would specify if ears are pointing then it would be cat or if ears are sloppy then it would be dog and other specification. More specification results into better computer judgement. But in case of ANN we would not give any specification rather we would give pictures tagged as a dog or a cat and let computer learn on its own.
So in case of ANN we just provide data with target variable already categorized or tagged and we let algorithm to decide on the output value depending upon latest information available. In this topic we would try understand how the develop ANN.
Let us try to understand with the help of a single layer ANN.
As we can see in the above figure, Neuron gets information from input from variables with weights being assigned to different variables. Neuron would summarize the input variables, apply the activation function and would get a result.
Now ANN would derive a cost function by comparing the predicted value with actual value. There are several ways we can define a cost function but the simplest one is:
Cost Function = ∑ ½(Predicted Value – Actual Value) ^2
ANN will try to minimize this Cost function. How we exactly achieve the minimum value can be understood with the help of concept back propagation (we would learn about back propagation in detail in following article).
- Once we get value for Cost Function, the information would go back into neural network
- Neural network will try to adjust the weights and recalculate the output value
- Again we would try to assess value of cost function
We would keep repeating above steps until value of cost function becomes very close to zero. The above algorithm is applied at row level.
As we can see if we have 1000 rows in our data, then ANN would try to compute the cost function by summarizing difference between predicted value and actual value for all the 1000 rows and adjust the weights. Basically all the rows would share the same weights once cost function minimization is achieved.
Next Section: Artificial Neural Network Part 6