Neural Network with Adaptive Learning Rate

Abstract

Over the last two decades, the neural network has surprisingly arisen as an efficient tool for dealing with numerous real-life applications. Optimization of the hyperparameter of the neural network attracted many researchers in industrial and research areas because of its great effect on the quality of the solution. This paper presents a new adaptation for the learning rate with shock (ALRS) as the learning rate is considered one of the most important hyperparameters. The experimental results proved that the new adaptation leads to improved accuracy with a simpler structure for the neural network regardless of the initial value of the learning rate.