General Bounds and Finite-Time Improvement for the Kiefer-Wolfowitz Stochastic Approximation Algorithm

Authors:
Mark Broadie;Deniz Cicek;Assaf Zeevi
Affiliations:
Graduate School of Business, Columbia University, New York, New York 10027;Graduate School of Business, Columbia University, New York, New York 10027;Graduate School of Business, Columbia University, New York, New York 10027
Venue:
Operations Research
Year:
2011

Citing 5
Cited 6

Adaptive algorithms and stochastic approximations

Adaptive algorithms and stochastic approximations
Acceleration of stochastic approximation by averaging

SIAM Journal on Control and Optimization
A scaled stochastic approximation algorithm

Management Science
Weighted Means in Stochastic Approximation of Minima

SIAM Journal on Control and Optimization
Robust Stochastic Approximation Approach to Stochastic Programming

SIAM Journal on Optimization

Dynamic Pricing Under a General Parametric Choice Model

Operations Research
Averaging and derivative estimation within stochastic approximation algorithms

Proceedings of the Winter Simulation Conference
An introspective on the retrospective-approximation paradigm

Proceedings of the Winter Simulation Conference
A Bayesian approach to stochastic root finding

Proceedings of the Winter Simulation Conference
A regularized adaptive steplength stochastic approximation scheme for monotone stochastic variational inequalities

Proceedings of the Winter Simulation Conference
Multidimensional stochastic approximation: Adaptive algorithms and applications

ACM Transactions on Modeling and Computer Simulation (TOMACS) - Special issue on simulation in complex service systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

We consider the Kiefer-Wolfowitz (KW) stochastic approximation algorithm and derive general upper bounds on its mean-squared error. The bounds are established using an elementary induction argument and phrased directly in the terms of tuning sequences of the algorithm. From this we deduce the nonnecessity of one of the main assumptions imposed on the tuning sequences by Kiefer and Wolfowitz [Kiefer, J., J. Wolfowitz. 1952. Stochastic estimation of the maximum of a regression function. Ann. Math. Statist.23(3) 462--466] and essentially all subsequent literature. The optimal choice of sequences is derived for various cases of interest, and an adaptive version of the KW algorithm, scaled-and-shifted KW (or SSKW), is proposed with the aim of improving its finite-time behavior. The key idea is to dynamically scale and shift the tuning sequences to better match them with characteristics of the unknown function and noise level, and thus improve algorithm performance. Numerical results are provided that illustrate that the proposed algorithm retains the convergence properties of the original KW algorithm while dramatically improving its performance in some cases.