Adaptive Nonlinear Discriminant Analysis by Regularized Minimum Squared Errors

Authors:
Hyunsoo Kim;Barry L. Drake;Haesun Park
Affiliations:
-;IEEE;-
Venue:
IEEE Transactions on Knowledge and Data Engineering
Year:
2006

Citing 20
Cited 3

A note on downdating the Cholesky factorization

SIAM Journal on Scientific and Statistical Computing
Introduction to statistical pattern recognition (2nd ed.)

Introduction to statistical pattern recognition (2nd ed.)
Fast adaptive condition estimation

SIAM Journal on Matrix Analysis and Applications
A singular value decomposition updating algorithm for subspace tracking

SIAM Journal on Matrix Analysis and Applications
Adaptive condition estimation for bank-one updates of QR factorizations

SIAM Journal on Matrix Analysis and Applications
Updating a rank-revealing ULV decomposition

SIAM Journal on Matrix Analysis and Applications
Updating URV decompositions in parallel

Parallel Computing
Block Downdating of Least Squares Solutions

SIAM Journal on Matrix Analysis and Applications
Accurate Downdating of Least Squares Solutions

SIAM Journal on Matrix Analysis and Applications
Perturbation analysis for block downdating of a Cholesky decomposition

Numerische Mathematik
The nature of statistical learning theory

The nature of statistical learning theory
Matrix computations (3rd ed.)

Matrix computations (3rd ed.)
An introduction to support Vector Machines: and other kernel-based learning methods

An introduction to support Vector Machines: and other kernel-based learning methods
Updating a Generalized URV Decomposition

SIAM Journal on Matrix Analysis and Applications
Nonlinear Fisher discriminant analysis using a minimum squared error cost function and the orthogonal least squares algorithm

Neural Networks
Structure Preserving Dimension Reduction for Clustered Text Data Based on the Generalized Singular Value Decomposition

SIAM Journal on Matrix Analysis and Applications
Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)
Machine learning and bioinformatics

Machine learning and bioinformatics
Nonlinear Discriminant Analysis Using Kernel Functions and the Generalized Singular Value Decomposition

SIAM Journal on Matrix Analysis and Applications
A Relationship between Linear Discriminant Analysis and the Generalized Minimum Squared~Error~Solution

SIAM Journal on Matrix Analysis and Applications

From fisher's linear discriminant analysis to NLDA or the story of the solution of a very difficult nonlinear classification problem

SMO'06 Proceedings of the 6th WSEAS International Conference on Simulation, Modelling and Optimization
Softdoublemaxminover: perceptron-like training of support vector machines

IEEE Transactions on Neural Networks
Kernel-based nonlinear discriminant analysis using minimum squared errors criterion for multiclass and undersampled problems

Signal Processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Kernelized nonlinear extensions of Fisher's discriminant analysis, discriminant analysis based on generalized singular value decomposition (LDA/GSVD), and discriminant analysis based on the minimum squared error formulation (MSE) have recently been widely utilized for handling undersampled high-dimensional problems and nonlinearly separable data sets. As the data sets are modified from incorporating new data points and deleting obsolete data points, there is a need to develop efficient updating and downdating algorithms for these methods to avoid expensive recomputation of the solution from scratch. In this paper, an efficient algorithm for adaptive linear and nonlinear kernel discriminant analysis based on regularized MSE, called adaptive KDA/RMSE, is proposed. In adaptive KDA/RMSE, updating and downdating of the computationally expensive eigenvalue decomposition (EVD) or singular value decomposition (SVD) is approximated by updating and downdating of the QR decomposition achieving an order of magnitude speed up. This fast algorithm for adaptive kernelized discriminant analysis is designed by utilizing regularization techniques and the relationship between linear and nonlinear discriminant analysis and the MSE. In addition, an efficient algorithm to compute leave-one-out cross validation is also introduced by utilizing downdating of KDA/RMSE.