Relaxed Steepest Descent and Cauchy-Barzilai-Borwein Method

  • Authors:
  • Marcos Raydan;Benar F. Svaiter

  • Affiliations:
  • Dpto. de Computación, Facultad de Ciencias, Universidad Central de Venezuela, Ap. 47002, Caracas 1041-A, Venezuela. mraydan@reacciun.ve;Instituto de Matemática Pura e Aplicada, Estrada dona Castorina 110, Rio de Janeiro, RJ CEP 22460-320, Brazil. benar@impa.br

  • Venue:
  • Computational Optimization and Applications
  • Year:
  • 2002

Quantified Score

Hi-index 0.00

Visualization

Abstract

The negative gradient direction to find local minimizers has been associated with the classical steepest descent method which behaves poorly except for very well conditioned problems. We stress out that the poor behavior of the steepest descent methods is due to the optimal Cauchy choice of steplength and not to the choice of the search direction. We discuss over and under relaxation of the optimal steplength. In fact, we study and extend recent nonmonotone choices of steplength that significantly enhance the behavior of the method. For a new particular case (Cauchy-Barzilai-Borwein method), we present a convergence analysis and encouraging numerical results to illustrate the advantages of using nonmonotone overrelaxations of the gradient method.