Advanced search algorithms for information-theoretic learning with kernel-based estimators

Authors:
R. A. Morejon;J. C. Principe
Affiliations:
Comput. Neuroengineering Lab., Univ. of Florida, Gainesville, FL, USA;-
Venue:
IEEE Transactions on Neural Networks
Year:
2004

Citing 0
Cited 9

ANN-based estimator for distillation using Levenberg-Marquardt approach

Engineering Applications of Artificial Intelligence
A minimum-error entropy criterion with self-adjusting step-size (MEE-SAS)

Signal Processing
Neural systems with numerically matched input-output statistic: isotonic bivariate statistical modeling

Computational Intelligence and Neuroscience - EEG/MEG Signal Processing
Letter: A robust approach to empirical PDF estimate

Neurocomputing
Information theoretic learning with adaptive kernels

Signal Processing
Mean-square convergence analysis of ADALINE training with minimum error entropy criterion

IEEE Transactions on Neural Networks
Supposed maximum information for comprehensible representations in SOM

Neurocomputing
Relative information maximization and its application to the extraction of explicit class structure in SOM

Neurocomputing
A new histogram-based estimation technique of entropy and mutual information using mean squared error minimization

Computers and Electrical Engineering

Quantified Score

Hi-index	0.00

Visualization

Abstract

Recent publications have proposed various information-theoretic learning (ITL) criteria based on Renyi's quadratic entropy with nonparametric kernel-based density estimation as alternative performance metrics for both supervised and unsupervised adaptive system training. These metrics, based on entropy and mutual information, take into account higher order statistics unlike the mean-square error (MSE) criterion. The drawback of these information-based metrics is the increased computational complexity, which underscores the importance of efficient training algorithms. In this paper, we examine familiar advanced-parameter search algorithms and propose modifications to allow training of systems with these ITL criteria. The well known algorithms tailored here for ITL include various improved gradient-descent methods, conjugate gradient approaches, and the Levenberg-Marquardt (LM) algorithm. Sample problems and metrics are presented to illustrate the computational efficiency attained by employing the proposed algorithms.