The Incremental Gauss-Newton Algorithm with Adaptive Stepsize Rule

Authors:
Hiroyuki Moriyama;Nobuo Yamashita;Masao Fukushima
Affiliations:
Department of Applied Mathematics and Physics, Graduate School of Informatics, Kyoto University, Kyoto 606-8501, Japan;Department of Applied Mathematics and Physics, Graduate School of Informatics, Kyoto University, Kyoto 606-8501, Japan;Department of Applied Mathematics and Physics, Graduate School of Informatics, Kyoto University, Kyoto 606-8501, Japan
Venue:
Computational Optimization and Applications
Year:
2003

Citing 6
Cited 1

Learning internal representations by error propagation

Parallel distributed processing: explorations in the microstructure of cognition, vol. 1
On the convergence of the LMS algorithm with adaptive learning rate for linear feedforward networks

Neural Computation
Incremental Least Squares Methods and the Extended Kalman Filter

SIAM Journal on Optimization
An Incremental Gradient(-Projection) Method with Momentum Term and Adaptive Stepsize Rule

SIAM Journal on Optimization
A New Class of Incremental Gradient Methods for Least Squares Problems

SIAM Journal on Optimization
Numerical Methods for Unconstrained Optimization and Nonlinear Equations (Classics in Applied Mathematics, 16)

Numerical Methods for Unconstrained Optimization and Nonlinear Equations (Classics in Applied Mathematics, 16)

A recursive algorithm for nonlinear least-squares problems

Computational Optimization and Applications

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we consider the Extended Kalman Filter (EKF) for solving nonlinear least squares problems. EKF is an incremental iterative method based on Gauss-Newton method that has nice convergence properties. Although EKF has the global convergence property under some conditions, the convergence rate is only sublinear under the same conditions. One of the reasons why EKF shows slow convergence is the lack of explicit stepsize. In the paper, we propose a stepsize rule for EKF and establish global convergence of the algorithm under the boundedness of the generated sequence and appropriate assumptions on the objective function. A notable feature of the stepsize rule is that the stepsize is kept greater than or equal to 1 at each iteration, and increases at a linear rate of k under an additional condition. Therefore, we can expect that the proposed method converges faster than the original EKF. We report some numerical results, which demonstrate that the proposed method is promising.