Mean-square convergence analysis of ADALINE training with minimum error entropy criterion

Authors:
Badong Chen;Yu Zhu;Jinchun Bu
Affiliations:
Department of Precision Instruments and Mechanology, Institute of Manufacturing Engineering, Tsinghua University, Beijing, China;Department of Precision Instruments and Mechanology, Institute of Manufacturing Engineering, Tsinghua University, Beijing, China;Nanjing University of Aeronautics and Astronautics, Nanjing, China and Tsinghua University, Beijing, China
Venue:
IEEE Transactions on Neural Networks
Year:
2010

Citing 15
Cited 0

Elements of information theory

Elements of information theory
Adaptive filter theory (3rd ed.)

Adaptive filter theory (3rd ed.)
Adaptive filters with error nonlinearities: mean-square analysis and optimum design

EURASIP Journal on Applied Signal Processing - Nonlinear signal and image processing - part I
Learning from Examples with Information Theoretic Criteria

Journal of VLSI Signal Processing Systems
A minimum-error entropy criterion with self-adjusting step-size (MEE-SAS)

Signal Processing
Diffusion Least-Mean Squares Over Adaptive Networks: Formulation and Performance Analysis

IEEE Transactions on Signal Processing - Part II
Convergence properties and data efficiency of the minimum error entropy criterion in ADALINE training

IEEE Transactions on Signal Processing
An error-entropy minimization algorithm for supervised training ofnonlinear adaptive systems

IEEE Transactions on Signal Processing
A unified approach to the steady-state and tracking analyses ofadaptive filters

IEEE Transactions on Signal Processing
Transient analysis of data-normalized adaptive filters

IEEE Transactions on Signal Processing
Transient analysis of adaptive filters with error nonlinearities

IEEE Transactions on Signal Processing
Mean-square performance of a family of affine projection algorithms

IEEE Transactions on Signal Processing
Generalized information potential criterion for adaptive system training

IEEE Transactions on Neural Networks
Advanced search algorithms for information-theoretic learning with kernel-based estimators

IEEE Transactions on Neural Networks
Continuously Differentiable Sample-Spacing Entropy Estimation

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.01

Visualization

Abstract

Recently, the minimum error entropy (MEE) criterion has been used as an information theoretic alternative to traditional mean-square error criterion in supervised learning systems. MEE yields nonquadratic, nonconvex performance surface even for adaptive linear neuron (ADALINE) training, which complicates the theoretical analysis of the method. In this paper, we develop a unified approach for mean-square convergence analysis for ADALINE training under MEE criterion. The weight update equation is formulated in the form of block-data. Based on a block version of energy conservation relation, and under several assumptions, we carry out the mean-square convergence analysis of this class of adaptation algorithm, including mean-square stability, mean-square evolution (transient behavior) and the mean-square steady-state performance. Simulation experimental results agree with the theoretical predictions very well.