Optimal design of regularization term and regularization parameter by subspace information criterion

Authors:
Masashi Sugiyama;Hidemitsu Ogawa
Affiliations:
Department of Computer Science, Graduate School of Information Science and Engineering, Tokyo Institute of Technology, 2-12-1 O-okayama, Meguro-ku, Tokyo 152-8552, Japan;Department of Computer Science, Graduate School of Information Science and Engineering, Tokyo Institute of Technology, 2-12-1 O-okayama, Meguro-ku, Tokyo 152-8552, Japan
Venue:
Neural Networks
Year:
2002

Citing 9
Cited 7

Statistical aspects of model selection

From data to model
Bayesian interpolation

Neural Computation
The nature of statistical learning theory

The nature of statistical learning theory
Theoretical and Experimental Evaluation of the Subspace Information Criterion

Machine Learning
An Adaptive Regularization Criterion for Supervised Learning

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Regularization Learning and Early Stopping in Linear Networks

IJCNN '00 Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks (IJCNN'00)-Volume 4 - Volume 4
Subspace Information Criterion for Model Selection

Neural Computation
Algebraic Analysis for Nonidentifiable Learning Machines

Neural Computation
Model complexity control for regression using VC generalization bounds

IEEE Transactions on Neural Networks

The subspace information criterion for infinite dimensional hypothesis spaces

The Journal of Machine Learning Research
Trading variance reduction with unbiasedness: the regularized subspace information criterion for robust model selection in kernel regression

Neural Computation
Matrix-pattern-oriented Ho-Kashyap classifier with regularization learning

Pattern Recognition
On objective function, regularizer, and prediction error of a learning algorithm for dealing with multiplicative weight noise

IEEE Transactions on Neural Networks
Convergence and objective functions of some fault/noise-injection-based online learning algorithms for RBF networks

IEEE Transactions on Neural Networks
Semi-supervised learning based on high density region estimation

Neural Networks
On the selection of weight decay parameter for faulty networks

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

The problem of designing the regularization term and regularization parameter for linear regression models is discussed. Previously, we derived an approximation to the generalization error called the subspace information criterion (SIC), which is an unbiased estimator of the generalization error with finite samples under certain conditions. In this paper, we apply SIC to regularization learning and use it for: (a) choosing the optimal regularization term and regularization parameter from the given candidates; (b) obtaining the closed form of the optimal regularization parameter for a fixed regularization term. The effectiveness of SIC is demonstrated through computer simulations with artificial and real data.