SVM decision boundary based discriminative subspace induction

Authors:
Jiayong Zhang;Yanxi Liu
Affiliations:
The Robotics Institute, Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213, USA;The Robotics Institute, Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213, USA
Venue:
Pattern Recognition
Year:
2005

Citing 21
Cited 0

Algorithms for clustering data

Algorithms for clustering data
Introduction to statistical pattern recognition (2nd ed.)

Introduction to statistical pattern recognition (2nd ed.)
Nonlinear component analysis as a kernel eigenvalue problem

Neural Computation
Heteroscedastic discriminant analysis and reduced rank HMMs for improved speech recognition

Speech Communication
Classifying Facial Actions

IEEE Transactions on Pattern Analysis and Machine Intelligence
Statistical Pattern Recognition: A Review

IEEE Transactions on Pattern Analysis and Machine Intelligence
Fractional-Step Dimensionality Reduction

IEEE Transactions on Pattern Analysis and Machine Intelligence
PCA versus LDA

IEEE Transactions on Pattern Analysis and Machine Intelligence
Multiclass Linear Dimension Reduction by Weighted Pairwise Fisher Criteria

IEEE Transactions on Pattern Analysis and Machine Intelligence
Complexity Measures of Supervised Classification Problems

IEEE Transactions on Pattern Analysis and Machine Intelligence
Gene Selection for Cancer Classification using Support Vector Machines

Machine Learning
Feature Extraction Based on Decision Boundaries

IEEE Transactions on Pattern Analysis and Machine Intelligence
Toward Bayes-Optimal Linear Dimension Reduction

IEEE Transactions on Pattern Analysis and Machine Intelligence
Filters, Wrappers and a Boosting-Based Hybrid for Feature Selection

ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Experiments with Random Projection

UAI '00 Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence
Linear Feature Extractors Based on Mutual Information

ICPR '96 Proceedings of the 13th International Conference on Pattern Recognition - Volume 2
Maximum likelihood discriminant feature spaces

ICASSP '00 Proceedings of the Acoustics, Speech, and Signal Processing, 2000. on IEEE International Conference - Volume 02
Nonparametric Discriminant Analysis

IEEE Transactions on Pattern Analysis and Machine Intelligence
Pattern recognition using discriminative feature extraction

IEEE Transactions on Signal Processing
Nonparametric discriminant analysis via recursive optimization ofPatrick-Fisher distance

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Decision boundary feature extraction for neural networks

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.01

Visualization

Abstract

We study the problem of linear dimension reduction for classification, with a focus on sufficient dimension reduction, i.e., finding subspaces without loss of discrimination power. First, we formulate the concept of sufficient subspace for classification in parallel terms as for regression. Then we present a new method to estimate the smallest sufficient subspace based on an improvement of decision boundary analysis (DBA). The main idea is to combine DBA with support vector machines (SVM) to overcome the inherent difficulty of DBA in small sample size situations while keeping DBA's estimation simplicity. The compact representation of SVM boundary results in a significant gain in both speed and accuracy over previous DBA implementations. Alternatively, this technique can be viewed as a way to reduce the run-time complexity of SVM itself. Comparative experiments on one simulated and four real-world benchmark datasets highlight the superior performance of the proposed approach.