Online Feature Selection Algorithm with Bayesian l 1 Regularization

Authors:
Yunpeng Cai;Yijun Sun;Jian Li;Steve Goodison
Affiliations:
Department of Electrical and Computer Engineering,;Interdisciplinary Center for Biotechnology Research,;Department of Electrical and Computer Engineering,;Department of Surgery, University of Florida, Gainesville, USA FL 32610
Venue:
PAKDD '09 Proceedings of the 13th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining
Year:
2009

Citing 15
Cited 0

A practical approach to feature selection

ML92 Proceedings of the ninth international workshop on Machine learning
Bayesian interpolation

Neural Computation
Bayesian parameter estimation via variational methods

Statistics and Computing
Novel Methods for Subset Selection with Respect to Problem Knowledge

IEEE Intelligent Systems
Introduction to Stochastic Search and Optimization

Introduction to Stochastic Search and Optimization
An introduction to variable and feature selection

The Journal of Machine Learning Research
Margin based feature selection - theory and algorithms

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Feature selection, L1 vs. L2 regularization, and rotational invariance

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Online Selection of Discriminative Tracking Features

IEEE Transactions on Pattern Analysis and Machine Intelligence
Pattern Recognition and Machine Learning (Information Science and Statistics)

Pattern Recognition and Machine Learning (Information Science and Statistics)
Improved breast cancer prognosis through the combination of clinical and genetic markers

Bioinformatics
Iterative RELIEF for Feature Weighting: Algorithms, Theories, and Applications

IEEE Transactions on Pattern Analysis and Machine Intelligence
The evidence framework applied to classification networks

Neural Computation
The evidence framework applied to sparse kernel logistic regression

Neurocomputing
Similarity-based online feature selection in content-based image retrieval

IEEE Transactions on Image Processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

We propose a novel online-learning based feature selection algorithm for supervised learning in the presence of a huge amount of irrelevant features. The key idea of the algorithm is to decompose a nonlinear problem into a set of locally linear ones through local learning, and then estimate the relevance of features globally in a large margin framework with l 1 regularization. Unlike batch learning, the regularization parameter in online learning has to be tuned on-the-fly with the increasing of training data. We address this issue within the Bayesian learning paradigm, and provide an analytic solution for automatic estimation of the regularization parameter via variational methods. Numerical experiments on a variety of benchmark data sets are presented that demonstrate the effectiveness of the newly proposed feature selection algorithm.