A Generalized Accelerated Gradient Optimization Method

Abstract

In this paper, we extend the recently developed generalized heavy ball optimization algorithm by introducing an additional parameter. This yields an improved optimization algorithm which has a similar form to the triple momentum method. The global convergence of the proposed algorithm for a class of functions with sector-bounded gradients is established. This is achieved by using the circle criterion. The proposed algorithm is designed to have the best possible R-convergence rate consistent with global convergence established using the circle criterion.