Improving Search Efficiency of Incremental Variable Selection by Using Second-Order Optimal Criterion

Authors:
Kazumi Saito;Nobuaki Mutoh;Tetsuo Ikeda;Toshinao Goda;Kazuki Mochizuki
Affiliations:
School of Administration and Informatics, University of Shizuoka,;School of Administration and Informatics, University of Shizuoka,;School of Administration and Informatics, University of Shizuoka,;Department of Food and Nutritional Sciences, University of Shizuoka, Shizuoka, Japan 422-8526;Department of Food and Nutritional Sciences, University of Shizuoka, Shizuoka, Japan 422-8526
Venue:
KES '08 Proceedings of the 12th international conference on Knowledge-Based Intelligent Information and Engineering Systems, Part III
Year:
2008

Citing 7
Cited 0

Optimal brain damage

Advances in neural information processing systems 2
Neural Networks for Pattern Recognition

Neural Networks for Pattern Recognition
An introduction to variable and feature selection

The Journal of Machine Learning Research
Mlps (mono layer polynomials and multi layer perceptrons) for nonlinear modeling

The Journal of Machine Learning Research
Ranking a random feature for variable and feature selection

The Journal of Machine Learning Research
Learning to rank using gradient descent

ICML '05 Proceedings of the 22nd international conference on Machine learning
Computational Methods of Feature Selection (Chapman & Hall/Crc Data Mining and Knowledge Discovery Series)

Computational Methods of Feature Selection (Chapman & Hall/Crc Data Mining and Knowledge Discovery Series)

Quantified Score

Hi-index	0.00

Visualization

Abstract

We address the problem of improving search efficiency of incremental variable selection. As one application, we focus on generalized linear models that are linear with respect to their parameters, but their objective functions are not restricted to a standard sum of squared error. In this paper, we present a method for incrementally selecting a set of relevant variables together with a newly proposing criterion based on second-order optimality for our models. In our experiments using a synthetic dataset with tens of thousands of variables, we show that the proposed method was able to completely restore the relevant variables. Moreover, the method substantially improved the search efficiency in comparison to a conventional calculation method. Furthermore, it is shown that we obtained promissing initial results using a real dataset in health-checkup.