Supervised Learning
Supervised learning is based on training a data sample
from data source with correct classification already assigned.
Such techniques are utilized in feedforward or MultiLayer
Perceptron (MLP) models. These MLP has three distinctive
characteristics:
- One or more layers of hidden neurons that are not part of the input
or output layers of the network that enable the network to learn and
solve any complex problems
- The nonlinearity reflected in the neuronal activity is
differentiable and,
- The interconnection model of the network exhibits a high degree of
connectivity.
These characteristics along with learning through training
solve difficult and diverse problems. Learning through
training in a supervised ANN model also called as error backpropagation algorithm. The error correction-learning
algorithm trains the network based on the input-output
samples and finds error signal, which is the difference of the
output calculated and the desired output and adjusts the
synaptic weights of the neurons that is proportional to the
product of the error signal and the input instance of the
synaptic weight. Based on this principle, error back
propagation learning occurs in two passes:
Forward Pass:
Here, input vector is presented to the network. This input signal propagates forward, neuron by neuron through the network and emerges at the output end of
the network as output signal: y(n) = φ(v(n)) where v(n) is the induced local field of a neuron defined by v(n) =Σ w(n)y(n). The output that is calculated at the output layer o(n) is compared with the desired response d(n) and finds the error e(n) for that neuron. The synaptic weights of the network during this pass are remains same.
Backward Pass:
The error signal that is originated at the output neuron of that layer is propagated backward through network. This calculates the local gradient for each neuron in each layer and allows the synaptic weights of the network to undergo changes in accordance with the delta rule as:
Δw(n) = η * δ(n) * y(n).
This recursive computation is continued, with forward pass followed by the backward pass for each input pattern till the network is converged.
Supervised learning paradigm of an ANN is efficient and finds solutions to several linear and non-linear problems such as classification, plant control, forecasting, prediction, robotics etc.
Unsupervised Learning
Self-Organizing neural networks learn using unsupervised learning algorithm to identify hidden patterns in unlabelled input data. This unsupervised refers to the ability to learn and organize information without providing an error signal to evaluate the potential solution. The lack of direction for the learning algorithm in unsupervised learning can sometime be advantageous, since it lets the algorithm to look back for patterns that have not been previously considered. The main characteristics of Self-Organizing Maps (SOM) are:
- It transforms an incoming signal pattern of arbitrary dimension into
one or 2 dimensional map and perform this transformation adaptively
- The network represents feedforward structure with a single
computational layer consisting of neurons arranged in rows and
columns. At each stage of representation, each input signal is kept
in its proper context and,
- Neurons dealing with closely related pieces of information are close
together and they communicate through synaptic connections.
The computational layer is also called as competitive layer since the neurons in the layer compete with each other to become active. Hence, this learning algorithm is called competitive algorithm. Unsupervised algorithm in SOM
works in three phases:
Competition phase:
for each input pattern x, presented to the network, inner product with synaptic weight w is calculated and the neurons in the competitive layer finds a discriminant function that induce competition among the neurons and the synaptic weight vector that is close to the input vector in the Euclidean distance is announced as winner in the competition. That neuron is called best matching neuron,
i.e. x = arg min ║x - w║.
Cooperative phase:
the winning neuron determines the center of a topological neighborhood h of cooperating neurons. This is performed by the lateral interaction d among the
cooperative neurons. This topological neighborhood reduces its size over a time period.
Adaptive phase:
enables the winning neuron and its neighborhood neurons to increase their individual values of the discriminant function in relation to the input pattern
through suitable synaptic weight adjustments,
Δw = ηh(x)(x –w).
Upon repeated presentation of the training patterns, the synaptic weight vectors tend to follow the distribution of the input patterns due to the neighborhood updating and thus ANN learns without supervisor.
Self-Organizing Model naturally represents the neuro-biological behavior, and hence is used in many real world applications such as clustering, speech recognition, texture segmentation, vector coding etc.
Reference.
