Sample complexity of linear learning machines with different restrictions over weights

Authors:
Marcin Korzeń;Przemysław Klęsk
Affiliations:
Faculty of Computer Science and Information Technology, West Pomeranian University of Technology, Szczecin, Poland;Faculty of Computer Science and Information Technology, West Pomeranian University of Technology, Szczecin, Poland
Venue:
ICAISC'12 Proceedings of the 11th international conference on Artificial Intelligence and Soft Computing - Volume Part II
Year:
2012

Citing 11
Cited 0

Bayesian regularization and pruning using a Laplace prior

Neural Computation
Learning in Neural Networks: Theoretical Foundations

Learning in Neural Networks: Theoretical Foundations
A Tutorial on Support Vector Machines for Pattern Recognition

Data Mining and Knowledge Discovery
The Role of Occam‘s Razor in Knowledge Discovery

Data Mining and Knowledge Discovery
Covering number bounds of certain regularized linear function classes

The Journal of Machine Learning Research
Rademacher and gaussian complexities: risk bounds and structural results

The Journal of Machine Learning Research
Information Theory, Inference & Learning Algorithms

Information Theory, Inference & Learning Algorithms
Feature selection, L1 vs. L2 regularization, and rotational invariance

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Gene selection in cancer classification using sparse logistic regression with Bayesian regularization

Bioinformatics
An experimental test of Occam's razor in classification

Machine Learning
Sets of approximating functions with finite Vapnik-Chervonenkis dimension for nearest-neighbors algorithms

Pattern Recognition Letters

Quantified Score

Hi-index	0.00

Visualization

Abstract

Known are many different capacity measures for learning machines like: Vapnik-Chervonenkis dimension, covering numbers or fat dimension. In this paper we present experimental results of sample complexity estimation, taking into account rather simple learning machines linear in parameters. We show that, sample complexity can be quite different even for learning machines having the same VC-dimension. Moreover, independently from the capacity of a learning machine, the distribution of data is also significant. Experimental results are compared with known theoretical results for sample complexity and generalization bounds.