Gradient Methods with Adaptive Step-Sizes

  • Authors:
  • Bin Zhou;Li Gao;Yu-Hong Dai

  • Affiliations:
  • School of Mathematical Sciences and LMAM, Peking University, Beijing, People's Republic of China 100871;School of Mathematical Sciences and LMAM, Peking University, Beijing, People's Republic of China 100871;State Key Laboratory of Scientific and Engineering Computing, Institute of Computational Mathematics and Scientific/Engineering Computing, Academy of Mathematics and Systems Science, Chinese Acade ...

  • Venue:
  • Computational Optimization and Applications
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

Motivated by the superlinear behavior of the Barzilai-Borwein (BB) method for two-dimensional quadratics, we propose two gradient methods which adaptively choose a small step-size or a large step-size at each iteration. The small step-size is primarily used to induce a favorable descent direction for the next iteration, while the large step-size is primarily used to produce a sufficient reduction. Although the new algorithms are still linearly convergent in the quadratic case, numerical experiments on some typical test problems indicate that they compare favorably with the BB method and some other efficient gradient methods.