Accelerating the cubic regularization of Newton’s method on convex problems

Authors:
Yu. Nesterov
Affiliations:
Catholic University of Louvain (UCL), Center for Operations Research and Econometrics (CORE), 34 voie du Roman Pays, 1348, Louvain-la-Neuve, Belgium
Venue:
Mathematical Programming: Series A and B
Year:
2007

Citing 0
Cited 5

Global optimality conditions for cubic minimization problem with box or binary constraints

Journal of Global Optimization
Complexity bounds for second-order optimality in unconstrained optimization

Journal of Complexity
On the Complexity of Steepest Descent, Newton's and Regularized Newton's Methods for Nonconvex Unconstrained Optimization Problems

SIAM Journal on Optimization
A trust region algorithm with adaptive cubic regularization methods for nonsmooth convex minimization

Computational Optimization and Applications
On the Oracle Complexity of First-Order and Derivative-Free Algorithms for Smooth Nonconvex Minimization

SIAM Journal on Optimization

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper we propose an accelerated version of the cubic regularization of Newton’s method (Nesterov and Polyak, in Math Program 108(1): 177–205, 2006). The original version, used for minimizing a convex function with Lipschitz-continuous Hessian, guarantees a global rate of convergence of order $$O\big({1 \over k^2}\big)$$, where k is the iteration counter. Our modified version converges for the same problem class with order $$O\big({1 \over k^3}\big)$$, keeping the complexity of each iteration unchanged. We study the complexity of both schemes on different classes of convex problems. In particular, we argue that for the second-order schemes, the class of non-degenerate problems is different from the standard class.