Back propagation of accumulated errorThe algorithm is a neural network algorithm and a variant of the error back propagation algorithm. It is based on the update rule of minimizing the cumulative error and adjusts the parameters in the negative gradient direction of the target. The goal is to minimize the training error. It is also called the "back propagation algorithm". The multi-layer feedforward neural network trained using this algorithm is called the "back propagation network".

Error Backpropagation Error Backpropagation Algorithm (BP)

The error back propagation algorithm uses the forward neural network to calculate the training error, and then uses the training error to act back on the hidden layer neurons, thereby adjusting the connection weights and the threshold of each neuron, and through continuous updating, the training error is minimized.

Currently, most neural network training is based on the BP algorithm, which can be used not only for multi-layer feedforward neural networks, but also for recursive neural network training, etc. However, "BP network" generally refers to a multi-layer feedforward neural network trained with the BP algorithm.

Implementation

For each training example, the algorithm performs the following operations:

• First, the input example is provided to the input layer neurons, and then the signal is passed forward layer by layer until the output layer produces the result;
• Then the error of the output layer is calculated and then back-propagated to the hidden layer neurons;
• Finally, the connection weights and thresholds are adjusted according to the errors of the hidden layer neurons.

This iterative process continues until a certain stopping condition is reached.

Feedforward Neural Network (FP)

The feedforward neural network can be seen as a combination of multiple logistic regressions, except that the results can be obtained directly through hidden layer neurons. Its cost function is similar to the logistic function, except that it needs to sum over different categories.