Neural Networks and Back Propagation
Neural Networks and Back Propagation
Neural networks are established on computergenerated neurons, which are combined in a diversity of ways to form networks. Artificial Neural Network is programs intended to solve any problem by copying the structure and the function of our nervous system. Neural comes from neuron and network means they are connected as a graph. For a computing system to be called by these pretty names, the system needs to have a label directed graph structure where nodes complete some simple computations.
Biological Terminology 
Artificial Neural Network Terminology

Neuron  Node/Unit/Cell 
Synapse  Connection/Edge/Link 
Synaptic Efficiency  Connection Strength/Weight 
Firing frequency  Node output 
A neural network is similar to the human brain in the following two ways:
 A neural network acquires knowledge through learning.
 The knowledge of the neural network is stored inside the interconnection strengths called synaptic weight.
Types of Artificial Neural Network
There are many types of neural network, they are different from each other based on the network.
 Fully connected Neural network
 Layered Neural Network
 Feedforward Neural Network
 Radial basis function Neural Network
 Kohonen Self Organizing Neural Network
 Recurrent Neural Network(RNN)
 Convolutional Neural Network
 Acyclic Neural Network
 Modular Neural Network
1Fully Connected Network
A network in which every node is linked to every other node, and these connections can be either excitatory (positive weights), inhibitory (negative weights), or irrelevant (almost zero weights).
2Layered Network
These are networks in which nodes are partitioned into subsets called layers, with no connections from layer j to k if j > k.
3Acyclic Network
This is the subclass of the layered networks in which there are no intralayer connections. In other words, a connection may exist between any node in layer I and any node in layer j for i < j, but a connection is not allowed for i=j.
4Modular Neural Network
A lot of problems are solved by using neural networks whose architecture made by many modules, with light interconnections between them. However, Modules can be structured in many different ways as Hierarchical organization, Continuous refinement, Input modularity.
In ANNs, learning is the method of changing the weights of connections between the nodes of a listed network. Its architecture determines the learning ability of neural networks and by the algorithmic method chosen for training.
Supervised Learning
In supervised learning, the training data consist of pairs of input and desired output values that are traditionally represented in data vectors. Supervised learning can also be stated as classification, where we have a wide range of classifiers, (Multilayer perceptron, k nearest neighbor, etc.
Unsupervised Learning
In unsupervised methodology, no sample outputs are provided to the network against which it can measure its predictive performance for a given vector of inputs. The wellknown form of unsupervised learning is clustering where we group data in different clusters by their similarity.
Back Propagation Algorithm
The backpropagation algorithm (Rumelhart and McClelland, 1986) is used in layered feedforward Artificial Neural Networks. The backpropagation network is a type of multilayer feedforward, supervised learning network based on gradient descent learning rule. we provide an algorithm with examples of some inputs and outputs that we want the network to compute, and then the error (the difference between actual and expected results) is calculated. The idea of the backpropagation algorithm is to reduce this error until the Artificial Neural Network learns the training data.
The activation function of the artificial neurons in ANNs implementing the backpropagation algorithm is a weighted sum (the addition of the inputs xi multiplied by their weights Wji)
The bestknown output function is the sigmoidal function.
Since the error is the difference between the actual and the desired output, the error depends on the weights, and we have to adjust the weights for minimizing the error. We can also describe the error function for the output of each neuron.
Applications of Neural Network
 Image Processing and Pattern Recognition
 Character recognition
 Forecasting
 Speech Recognition
 Signature Verification
Advantages of Neural Network
 It involves humanlike thinking.
 Used in Data mining to handle noisy or missing data.
 Neural Networks can work through a large number of variables or parameters.
 They provide general solutions with good predictive accuracy.
 The system has got property of continuous learning.