On the Behavior of the Gradient Norm in the Steepest Descent Method

Authors:
Jorge Nocedal;Annick Sartenaer;Ciyou Zhu
Affiliations:
ECE Department, Northwestern University, Evanston, Il 60208, USA. nocedal@ece.nwu.edu, www.eecs.nwu.edu/~nocedal/PSfiles;Department of Mathematics, Facultés Universitaires Notre-Dame de la Paix, Namur, Belgium. Annick.Sartenaer@fundp.ac.be;Department of Electrical and Computer Engineering, Northwestern University, Evanston, Il 60208-3118, USA
Venue:
Computational Optimization and Applications
Year:
2002

Citing 3
Cited 5

CUTE: constrained and unconstrained testing environment

ACM Transactions on Mathematical Software (TOMS)
A limited memory algorithm for bound constrained optimization

SIAM Journal on Scientific Computing
Lancelot: A FORTRAN Package for Large-Scale Nonlinear Optimization (Release A)

Lancelot: A FORTRAN Package for Large-Scale Nonlinear Optimization (Release A)

A New Gradient Method with an Optimal Stepsize Property

Computational Optimization and Applications
A new trust region method with adaptive radius

Computational Optimization and Applications
A Line Search Multigrid Method for Large-Scale Nonlinear Optimization

SIAM Journal on Optimization
The Chaotic Nature of Faster Gradient Descent Methods

Journal of Scientific Computing
A globally optimal tri-vector method to solve an ill-posed linear system

Journal of Computational and Applied Mathematics

Quantified Score

Hi-index	0.00

Visualization

Abstract

It is well known that the norm of the gradient may be unreliable as a stopping test in unconstrained optimization, and that it often exhibits oscillations in the course of the optimization. In this paper we present results descibing the properties of the gradient norm for the steepest descent method applied to quadratic objective functions. We also make some general observations that apply to nonlinear problems, relating the gradient norm, the objective function value, and the path generated by the iterates.