Sparse multinomial kernel discriminant analysis (sMKDA)

Authors:
Robert F. Harrison;Kitsuchart Pasupa
Affiliations:
Department of Automatic Control and Systems Engineering, The University of Sheffield, Mappin Street, Sheffield S1 3JD, UK;School of Electronics and Computer Science, The University of Southampton, Southampton SO17 1BJ, UK
Venue:
Pattern Recognition
Year:
2009

Citing 22
Cited 2

A least squares approach to linear discriminant analysis

SIAM Journal on Scientific and Statistical Computing
A Recursive Orthogonal Least Squares Algorithm for Training RBF Networks

Neural Processing Letters
Nonlinear Fisher discriminant analysis using a minimum squared error cost function and the orthogonal least squares algorithm

Neural Networks
Bayesian framework for least-squares support vector machine classifiers, Gaussian processes, and kernel fisher discriminant analysis

Neural Computation
Constructing Descriptive and Discriminative Nonlinear Features: Rayleigh Coefficients in Kernel Feature Spaces

IEEE Transactions on Pattern Analysis and Machine Intelligence
Sparse bayesian learning and the relevance vector machine

The Journal of Machine Learning Research
Kernel Methods for Pattern Analysis

Kernel Methods for Pattern Analysis
In Defense of One-Vs-All Classification

The Journal of Machine Learning Research
A modified algorithm for generalized discriminant analysis

Neural Computation
Sparse Multinomial Logistic Regression: Fast Algorithms and Generalization Bounds

IEEE Transactions on Pattern Analysis and Machine Intelligence
Feature Scaling for Kernel Fisher Discriminant Analysis Using Leave-One-Out Cross Validation

Neural Computation
Generalized Discriminant Analysis Using a Kernel Approach

Neural Computation
Using discriminant analysis for multi-class classification: an experimental investigation

Knowledge and Information Systems
A fast kernel-based nonlinear discriminant analysis for multi-class problems

Pattern Recognition
Optimising Kernel Parameters and Regularisation Coefficients for Non-linear Discriminant Analysis

The Journal of Machine Learning Research
Sparse least squares support vector training in the reduced empirical feature space

Pattern Analysis & Applications
On kernel difference-weighted k-nearest neighbor classification

Pattern Analysis & Applications - Special Issue: Non-parametric distance-based classification techniques and their applications
Local regularization assisted orthogonal least squares regression

Neurocomputing
Rapid and brief communications: Kernel direct discriminant analysis and its theoretical foundation

Pattern Recognition
Sparse modeling using orthogonal forward regression with PRESS statistic and regularization

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
A comparison of methods for multiclass support vector machines

IEEE Transactions on Neural Networks
Face recognition using kernel direct discriminant analysis algorithms

IEEE Transactions on Neural Networks

Letters: Kernel-view based discriminant approach for embedded feature extraction in high-dimensional space

Neurocomputing
Regularized orthogonal linear discriminant analysis

Pattern Recognition

Quantified Score

Hi-index	0.01

Visualization

Abstract

Dimensionality reduction via canonical variate analysis (CVA) is important for pattern recognition and has been extended variously to permit more flexibility, e.g. by ''kernelizing'' the formulation. This can lead to over-fitting, usually ameliorated by regularization. Here, a method for sparse, multinomial kernel discriminant analysis (sMKDA) is proposed, using a sparse basis to control complexity. It is based on the connection between CVA and least-squares, and uses forward selection via orthogonal least-squares to approximate a basis, generalizing a similar approach for binomial problems. Classification can be performed directly via minimum Mahalanobis distance in the canonical variates. sMKDA achieves state-of-the-art performance in terms of accuracy and sparseness on 11 benchmark datasets.