Weighted feature extraction with a functional data extension

Authors:
Luis Gonzalo Sánchez Giraldo;Germán Castellanos Domínguez
Affiliations:
Electrical and Computer Engineering Department, University of Florida, Gainesville, USA;Electrical, Electronics and Computing Engineering Department, Universidad Nacional de Colombia, Manizales, Caldas, Colombia
Venue:
Neurocomputing
Year:
2010

Citing 15
Cited 1

Matrix analysis

Matrix analysis
The nature of statistical learning theory

The nature of statistical learning theory
Selection of relevant features and examples in machine learning

Artificial Intelligence - Special issue on relevance
EM algorithms for PCA and SPCA

NIPS '97 Proceedings of the 1997 conference on Advances in neural information processing systems 10
A Formalism for Relevance and Its Application in Feature Subset Selection

Machine Learning
Feature Selection Via Mathematical Programming

INFORMS Journal on Computing
An introduction to variable and feature selection

The Journal of Machine Learning Research
Efficient Feature Selection via Analysis of Relevance and Redundancy

The Journal of Machine Learning Research
Feature Selection for Unsupervised and Supervised Inference: The Emergence of Sparsity in a Weight-Based Approach

The Journal of Machine Learning Research
Nonparametric Functional Data Analysis: Theory and Practice (Springer Series in Statistics)

Nonparametric Functional Data Analysis: Theory and Practice (Springer Series in Statistics)
Prototype selection for dissimilarity-based classifiers

Pattern Recognition
Enhanced feature selection models using gradient-based and point injection techniques

Neurocomputing
Discriminative Locality Alignment

ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part I
Geometric Mean for Subspace Selection

IEEE Transactions on Pattern Analysis and Machine Intelligence
Automatic clinical image segmentation using pathological modelling, PCA and SVM

MLDM'05 Proceedings of the 4th international conference on Machine Learning and Data Mining in Pattern Recognition

Applying soft computing techniques to optimise a dental milling process

Neurocomputing

Quantified Score

Hi-index	0.01

Visualization

Abstract

Dimensionality reduction has proved to be a beneficial tool in learning problems. Two of the main advantages provided by dimensionality reduction are interpretation and generalization. Typically, dimensionality reduction is addressed in two separate ways: variable selection and feature extraction. However, in the recent years there has been a growing interest in developing combined schemes such as feature extraction with built-in feature selection. In this paper, we look at dimensionality reduction as a rank-deficient problem that embraces variable selection and feature extraction, simultaneously. From our analysis, we derive a weighting algorithm that is able to select and linearly transform variables by fixing the dimensionality of the space where a relevance criterion is evaluated. This step enforces sparseness on the resulting weights. Our main goal is dimensionality reduction for classification problems. Namely, we introduce modified versions of principal component analysis (PCA) by expectation maximization (EM) and linear regularized discriminant analysis (RDA). Finally, we propose a simple extension of WRDA that deals with functional features. In this case, observations are described by a set of functions defined over the same domain. Methods were put to test on artificial and real data sets showing high levels of generalization even for small sized training samples.