A class of stochastic gradient algorithms with exponentiated error cost functions

Authors:
C. Boukis;D. P. Mandic;A. G. Constantinides
Affiliations:
Athens Information Technology, Autonomic and Grid Computing Group, Markopoulo Ave, Peania/Athens 19002, Greece;Imperial College, Electrical and Electronic Engineering Department, Communications and Signal Processing Group, Exhibition Rd, London SW7 2BT, UK;Imperial College, Electrical and Electronic Engineering Department, Communications and Signal Processing Group, Exhibition Rd, London SW7 2BT, UK
Venue:
Digital Signal Processing
Year:
2009

Citing 11
Cited 3

Adaptive signal processing

Adaptive signal processing
Adaptive algorithms and stochastic approximations

Adaptive algorithms and stochastic approximations
Exponentiated gradient versus gradient descent for linear predictors

Information and Computation
Detection, Estimation, and Modulation Theory: Radar-Sonar Signal Processing and Gaussian Signals in Noise

Detection, Estimation, and Modulation Theory: Radar-Sonar Signal Processing and Gaussian Signals in Noise
Adaptive filters with error nonlinearities: mean-square analysis and optimum design

EURASIP Journal on Applied Signal Processing - Nonlinear signal and image processing - part I
A time-domain feedback analysis of filtered-error adaptive gradientalgorithms

IEEE Transactions on Signal Processing
Exploiting sparsity in adaptive filters

IEEE Transactions on Signal Processing
A unified approach to the steady-state and tracking analyses ofadaptive filters

IEEE Transactions on Signal Processing
Transient analysis of adaptive filters with error nonlinearities

IEEE Transactions on Signal Processing
An easy demonstration of the optimum value of the adaptationconstant in the LMS algorithm [FIR filter theory]

IEEE Transactions on Education
The least mean fourth (LMF) adaptive algorithm and its family

IEEE Transactions on Information Theory

Fast communication: The exponentiated convex variable step-size (ECVSS) algorithm

Signal Processing
A modified stochastic gradient based parameter estimation algorithm for dual-rate sampled-data systems

Digital Signal Processing
Adaptive mixture methods based on Bregman divergences

Digital Signal Processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

A novel class of stochastic gradient descent algorithms is introduced based on the minimisation of convex cost functions with exponential dependence on the adaptation error, instead of the conventional linear combinations of even moments. The derivation is supported by rigourous analysis of the necessary conditions for convergence, the steady state mean square error is calculated and the optimal solutions in the least exponential sense are derived. The normalisation of the associated step size is also considered in order to fully exploit the dynamics of the input signal. Simulation results support the analysis.