Adaptive Learning Rate Algorithms - Yoni Iny @ Upsolver (Eng)

Slides: The standard backpropagation algorithm shows a very slow rate of convergence and a high dependency on the value of the learning rate parameter . The shape of the multi dimensional error function for the majority of the applications usually presents irregularities which are difficult to handle using gradient descent technique with a fixed step size . Another serious problem is the existence of regions with local minima. If the search procedure enters such a region, the mini