Rademacher and gaussian complexities: risk bounds and structural results

Authors:
Peter L. Bartlett;Shahar Mendelson
Affiliations:
Research School of Information Sciences and Engineering, Australian National University, Canberra 0200, Australia;Research School of Information Sciences and Engineering, Australian National University, Canberra 0200, Australia
Venue:
The Journal of Machine Learning Research
Year:
2003

Citing 11
Cited 134

An Experimental and Theoretical Comparison of Model SelectionMethods

Machine Learning - Special issue on the eighth annual conference on computational learning theory, (COLT '95)
Generalization in decision trees and DNF: does size matter?

NIPS '97 Proceedings of the 1997 conference on Advances in neural information processing systems 10
Improved Generalization Through Explicit Optimization of Margins

Machine Learning
An introduction to support Vector Machines: and other kernel-based learning methods

An introduction to support Vector Machines: and other kernel-based learning methods
Learning in Neural Networks: Theoretical Foundations

Learning in Neural Networks: Theoretical Foundations
Model Selection and Error Estimation

Machine Learning
Using output codes to boost multiclass learning problems

ICML '97 Proceedings of the Fourteenth International Conference on Machine Learning
The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network

IEEE Transactions on Information Theory
Structural risk minimization over data-dependent hierarchies

IEEE Transactions on Information Theory
Rademacher penalties and structural risk minimization

IEEE Transactions on Information Theory
Rademacher averages and phase transitions in Glivenko-Cantelli classes

IEEE Transactions on Information Theory

An introduction to boosting and leveraging

Advanced lectures on machine learning
Quantum optimization for training support vector machines

Neural Networks - 2003 Special issue: Advances in neural networks research — IJCNN'03
Data-dependent margin-based generalization bounds for classification

The Journal of Machine Learning Research
Greedy algorithms for classification—consistency, convergence rates, and adaptivity

The Journal of Machine Learning Research
Generalization error bounds for Bayesian mixture algorithms

The Journal of Machine Learning Research
On the rate of convergence of regularized boosting classifiers

The Journal of Machine Learning Research
Learning the Kernel Matrix with Semidefinite Programming

The Journal of Machine Learning Research
Distance--Based Classification with Lipschitz Functions

The Journal of Machine Learning Research
On Robustness Properties of Convex Risk Minimization Methods for Pattern Recognition

The Journal of Machine Learning Research
Selective Rademacher Penalization and Reduced Error Pruning of Decision Trees

The Journal of Machine Learning Research
Diffusion Kernels on Statistical Manifolds

The Journal of Machine Learning Research
On the Generalization Ability of GRLVQ Networks

Neural Processing Letters
Sparse Multinomial Logistic Regression: Fast Algorithms and Generalization Bounds

IEEE Transactions on Pattern Analysis and Machine Intelligence
Maximal margin classification for metric spaces

Journal of Computer and System Sciences - Special issue: Learning theory 2003
Simpler knowledge-based support vector machines

ICML '06 Proceedings of the 23rd international conference on Machine learning
Margin-based active learning for LVQ networks

Neurocomputing
Synthesis of maximum margin and multiview learning using unlabeled data

Neurocomputing
On nonparametric classification with missing covariates

Journal of Multivariate Analysis
Nonparametric Quantile Estimation

The Journal of Machine Learning Research
Consistency of Multiclass Empirical Risk Minimization Methods Based on Convex Loss

The Journal of Machine Learning Research
Learnability of Gaussians with Flexible Variances

The Journal of Machine Learning Research
Sample compression bounds for decision trees

Proceedings of the 24th international conference on Machine learning
Complexity of pattern classes and the Lipschitz property

Theoretical Computer Science
Incentive compatible regression learning

Proceedings of the nineteenth annual ACM-SIAM symposium on Discrete algorithms
The value of agreement a new boosting algorithm

Journal of Computer and System Sciences
A theory of learning with similarity functions

Machine Learning
Estimating labels from label proportions

Proceedings of the 25th international conference on Machine learning
SVM optimization: inverse dependence on training set size

Proceedings of the 25th international conference on Machine learning
Tailoring density estimation via reproducing kernel moment matching

Proceedings of the 25th international conference on Machine learning
Learning Similarity with Operator-valued Large-margin Classifiers

The Journal of Machine Learning Research
A Hilbert Space Embedding for Distributions

ALT '07 Proceedings of the 18th international conference on Algorithmic Learning Theory
Catenary Support Vector Machines

ECML PKDD '08 Proceedings of the 2008 European Conference on Machine Learning and Knowledge Discovery in Databases - Part I
Support Vector Machines, Data Reduction, and Approximate Kernel Matrices

ECML PKDD '08 Proceedings of the European conference on Machine Learning and Knowledge Discovery in Databases - Part II
Generalization Bounds for K-Dimensional Coding Schemes in Hilbert Spaces

ALT '08 Proceedings of the 19th international conference on Algorithmic Learning Theory
Structural Support Vector Machine

ISNN '08 Proceedings of the 5th international symposium on Neural Networks: Advances in Neural Networks
Learning from Multiple Sources

The Journal of Machine Learning Research
Boosting method for local learning in statistical pattern recognition

Neural Computation
Reducing mechanism design to algorithm design via machine learning

Journal of Computer and System Sciences
Learning rates of gradient descent algorithm for classification

Journal of Computational and Applied Mathematics
Convergence analysis of kernel Canonical Correlation Analysis: theory and practice

Machine Learning
Error bounds of multi-graph regularized semi-supervised classification

Information Sciences: an International Journal
Transfer bounds for linear feature learning

Machine Learning
The generalization performance of ERM algorithm with strongly mixing observations

Machine Learning
Multiple indefinite kernel learning with mixed norm regularization

ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Generalization analysis of listwise learning-to-rank algorithms

ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Robust support vector machine training via convex outlier ablation

AAAI'06 Proceedings of the 21st national conference on Artificial intelligence - Volume 1
Capacity Control for Partially Ordered Feature Sets

ECML PKDD '09 Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases: Part II
Learning to Rank for Information Retrieval

Foundations and Trends in Information Retrieval
Explanation-based feature construction

IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Error bounds of decision templates and support vector machines in decision fusion

International Journal of Knowledge Engineering and Soft Data Paradigms
Transductive Rademacher complexity and its applications

Journal of Artificial Intelligence Research
Large margin Boltzmann machines

IJCAI'09 Proceedings of the 21st international jont conference on Artifical intelligence
Learning SVMs from Sloppily Labeled Data

ICANN '09 Proceedings of the 19th International Conference on Artificial Neural Networks: Part I
A discriminative model for semi-supervised learning

Journal of the ACM (JACM)
Regularization techniques and suboptimal solutions to optimization problems in learning from data

Neural Computation
Adaptive relevance matrices in learning vector quantization

Neural Computation
Robustness and Regularization of Support Vector Machines

The Journal of Machine Learning Research
Margin-based Ranking and an Equivalence between AdaBoost and RankBoost

The Journal of Machine Learning Research
Estimating Labels from Label Proportions

The Journal of Machine Learning Research
Classification Using Geometric Level Sets

The Journal of Machine Learning Research
Maximum Relative Margin and Data-Dependent Regularization

The Journal of Machine Learning Research
Deriving the kernel from training data

MCS'07 Proceedings of the 7th international conference on Multiple classifier systems
Transductive rademacher complexity and its applications

COLT'07 Proceedings of the 20th annual conference on Learning theory
How good is a kernel when used as a similarity measure?

COLT'07 Proceedings of the 20th annual conference on Learning theory
Rademacher margin complexity

COLT'07 Proceedings of the 20th annual conference on Learning theory
Using unsupervised analysis to constrain generalization bounds for support vector classifiers

IEEE Transactions on Neural Networks
Gradient descent optimization of smoothed information retrieval metrics

Information Retrieval
Incentive compatible regression learning

Journal of Computer and System Sciences
Chromatic PAC-Bayes Bounds for Non-IID Data: Applications to Ranking and Stationary β-Mixing Processes

The Journal of Machine Learning Research
Rademacher chaos complexities for learning the kernel problem

Neural Computation
Generalized derivative based kernelized learning vector quantization

IDEAL'10 Proceedings of the 11th international conference on Intelligent data engineering and automated learning
On the power of topological kernel in microarray-based detection of cancer

IDEAL'10 Proceedings of the 11th international conference on Intelligent data engineering and automated learning
Rademacher complexity and grammar induction algorithms: what it may (not) tell us

ICGI'10 Proceedings of the 10th international colloquium conference on Grammatical inference: theoretical results and applications
A unifying view of multiple kernel learning

ECML PKDD'10 Proceedings of the 2010 European conference on Machine learning and knowledge discovery in databases: Part II
K-dimensional coding schemes in Hilbert spaces

IEEE Transactions on Information Theory
Sparse Semi-supervised Learning Using Conjugate Functions

The Journal of Machine Learning Research
On Convergence of Kernel Learning Estimators

SIAM Journal on Optimization
Maximal Discrepancy for Support Vector Machines

Neurocomputing
A novel multi-view classifier based on Nyström approximation

Expert Systems with Applications: An International Journal
Support Vector Machines with the Ramp Loss and the Hard Margin Loss

Operations Research
A novel multi-view learning developed from single-view patterns

Pattern Recognition
lp-Norm Multiple Kernel Learning

The Journal of Machine Learning Research
Learning from Partial Labels

The Journal of Machine Learning Research
A general framework for dimensionality reduction for large data sets

WSOM'11 Proceedings of the 8th international conference on Advances in self-organizing maps
Transfer learning with adaptive regularizers

ECML PKDD'11 Proceedings of the 2011 European conference on Machine learning and knowledge discovery in databases - Volume Part III
Sets of approximating functions with finite Vapnik-Chervonenkis dimension for nearest-neighbors algorithms

Pattern Recognition Letters
The consistency analysis of coefficient regularized classification with convex loss

WSEAS Transactions on Mathematics
The rademacher complexity of linear transformation classes

COLT'06 Proceedings of the 19th annual conference on Learning Theory
The value of agreement, a new boosting algorithm

COLT'05 Proceedings of the 18th annual conference on Learning Theory
Generalization error bounds using unlabeled data

COLT'05 Proceedings of the 18th annual conference on Learning Theory
Generalization behaviour of alkemic decision trees

ILP'05 Proceedings of the 15th international conference on Inductive Logic Programming
A general framework for dimensionality-reducing data visualization mapping

Neural Computation
Learning vector quantization classification with local relevance determination for medical data

ICAISC'06 Proceedings of the 8th international conference on Artificial Intelligence and Soft Computing
Generalization bounds for subspace selection and hyperbolic PCA

SLSFS'05 Proceedings of the 2005 international conference on Subspace, Latent Structure and Feature Selection
Multi-view laplacian support vector machines

ADMA'11 Proceedings of the 7th international conference on Advanced Data Mining and Applications - Volume Part II
Structured sparsity and generalization

The Journal of Machine Learning Research
A kernel two-sample test

The Journal of Machine Learning Research
Bound the learning rates with generalized gradients

WSEAS Transactions on Signal Processing
Generalization error bounds for the logical analysis of data

Discrete Applied Mathematics
A comparison of complexity selection approaches for polynomials based on: vapnik-chervonenkis dimension, rademacher complexity and covering numbers

ICAISC'12 Proceedings of the 11th international conference on Artificial Intelligence and Soft Computing - Volume Part II
Sample complexity of linear learning machines with different restrictions over weights

ICAISC'12 Proceedings of the 11th international conference on Artificial Intelligence and Soft Computing - Volume Part II
Application of integral operator for regularized least-square regression

Mathematical and Computer Modelling: An International Journal
On the performance of regularized regression learning in Hilbert space

Neurocomputing
Learning linear and kernel predictors with the 0-1 loss function

IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Three
A review of optimization methodologies in support vector machines

Neurocomputing
Fast Structured Prediction Using Large Margin Sigmoid Belief Networks

International Journal of Computer Vision
Learning Kernel-Based Halfspaces with the 0-1 Loss

SIAM Journal on Computing
Regularization techniques for learning with matrices

The Journal of Machine Learning Research
Regularized multi-view learning machine based on response surface technique

Neurocomputing
Multisource domain adaptation and its application to early detection of fatigue

ACM Transactions on Knowledge Discovery from Data (TKDD) - Special Issue on the Best of SIGKDD 2011
Learning the coordinate gradients

Advances in Computational Mathematics
An alternative approach to avoid overfitting for surrogate models

Proceedings of the Winter Simulation Conference
A Computational Learning Theory of Active Object Recognition Under Uncertainty

International Journal of Computer Vision
Effect on generalization of using relational information in list-wise algorithms

ICPCA/SWS'12 Proceedings of the 2012 international conference on Pervasive Computing and the Networked World
Learning with boundary conditions

Neural Computation
Error analysis of coefficient-based regularized algorithm for density-level detection

Neural Computation
Querying discriminative and representative samples for batch mode active learning

Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining
Multiple spectral kernel learning and a gaussian complexity computation

Neural Computation
On the convergence rate of lp-norm multiple kernel learning

The Journal of Machine Learning Research
Multi-instance learning with any hypothesis class

The Journal of Machine Learning Research
PAC-bayes bounds with data dependent priors

The Journal of Machine Learning Research
Eigenvalues perturbation of integral operator for kernel selection

Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Universal learning using free multivariate splines

Neurocomputing
Generalization performance of support vector classifiers for density level detection

Neurocomputing
Manifold-preserving graph reduction for sparse semi-supervised learning

Neurocomputing
Multi class learning with individual sparsity

IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Unlabeled patterns to tighten Rademacher complexity error bounds for kernel classifiers

Pattern Recognition Letters
Machine learning with operational costs

The Journal of Machine Learning Research
Distribution-dependent sample complexity of large margin learning

The Journal of Machine Learning Research
Learning bilinear model for matching queries and documents

The Journal of Machine Learning Research
Maximum volume clustering: a new discriminative clustering approach

The Journal of Machine Learning Research
Guaranteed classification via regularized similarity learning

Neural Computation
Learning vector quantization for (dis-)similarities

Neurocomputing
Approximation and Estimation Bounds for Subsets of Reproducing Kernel Kreĭn Spaces

Neural Processing Letters

Quantified Score

Hi-index	0.06

Visualization

Abstract

We investigate the use of certain data-dependent estimates of the complexity of a function class, called Rademacher and Gaussian complexities. In a decision theoretic setting, we prove general risk bounds in terms of these complexities. We consider function classes that can be expressed as combinations of functions from basis classes and show how the Rademacher and Gaussian complexities of such a function class can be bounded in terms of the complexity of the basis classes. We give examples of the application of these techniques in finding data-dependent risk bounds for decision trees, neural networks and support vector machines.