The discriminating power of random features

Authors:
Stefano Rovetta;Francesco Masulli;Maurizio Filippone
Affiliations:
Department of Computer and Information Sciences, University of Genova, Italy;Department of Computer and Information Sciences, University of Genova, Italy and Temple University, Philadelphia, USA;Sheffield University, UK
Venue:
Proceedings of the 2009 conference on Neural Nets WIRN09: Proceedings of the 19th Italian Workshop on Neural Nets, Vietri sul Mare, Salerno, Italy, May 28--30 2009
Year:
2009

Citing 8
Cited 0

The nature of statistical learning theory

The nature of statistical learning theory
Selection of relevant features and examples in machine learning

Artificial Intelligence - Special issue on relevance
An experimental comparison of naive Bayesian and keyword-based anti-spam filtering with personal e-mail messages

SIGIR '00 Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval
Automatic Construction of Decision Trees from Data: A Multi-Disciplinary Survey

Data Mining and Knowledge Discovery
Gene Selection for Cancer Classification using Support Vector Machines

Machine Learning
An introduction to variable and feature selection

The Journal of Machine Learning Research
Unsupervised gene selection and clustering using simulated annealing

WILF'05 Proceedings of the 6th international conference on Fuzzy Logic and Applications
Circular backpropagation networks for classification

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

Input selection is found as a part of several machine learning tasks, either to improve performance or as the main goal. For instance, gene selection in bioinformatics is an input selection problem. However, as we prove in this paper, the reliability of input selection in the presence of high-dimensional data is affected by a small-sample problem. As a consequence of this effect, even completely random inputs have a chance to be selected as very useful, even if they are not relevant from the point of view of the underlying model. We express the probability of this event as a function of data cardinality and dimensionality, discuss the applicability of this analysis, and compute the probability for some data sets. We also show, as an illustration, some experimental results obtained by applying a specific input selection algorithm, previously presented by the authors, which show how inputs known to be random are consistently selected by the method.