The Strength of Weak Learnability
Machine Learning
Acceleration of stochastic approximation by averaging
SIAM Journal on Control and Optimization
Boosting a weak learning algorithm by majority
Information and Computation
Exponentiated gradient versus gradient descent for linear predictors
Information and Computation
Proximal Minimization Methods with Generalized Bregman Functions
SIAM Journal on Control and Optimization
Relative Loss Bounds for Multidimensional Regression Problems
Machine Learning
The Ordered Subsets Mirror Descent Optimization Method with Applications to Tomography
SIAM Journal on Optimization
Solving large scale linear prediction problems using stochastic gradient descent algorithms
ICML '04 Proceedings of the twenty-first international conference on Machine learning
A Second-Order Perceptron Algorithm
SIAM Journal on Computing
IEEE Transactions on Signal Processing
On the generalization ability of on-line learning algorithms
IEEE Transactions on Information Theory
Mirror descent and nonlinear projected subgradient methods for convex optimization
Operations Research Letters
Relative loss bounds for single neurons
IEEE Transactions on Neural Networks
Aggregation by exponential weighting and sharp oracle inequalities
COLT'07 Proceedings of the 20th annual conference on Learning theory
Suboptimality of penalized empirical risk minimization in classification
COLT'07 Proceedings of the 20th annual conference on Learning theory
NL-Means and aggregation procedures
ICIP'09 Proceedings of the 16th IEEE international conference on Image processing
Dual Averaging Methods for Regularized Stochastic Learning and Online Optimization
The Journal of Machine Learning Research
Automation and Remote Control
Hyper-Sparse Optimal Aggregation
The Journal of Machine Learning Research
| Hi-index | 0.00 |
We consider a recursive algorithm to construct an aggregated estimator from a finite number of base decision rules in the classification problem. The estimator approximately minimizes a convex risk functional under the 驴1-constraint. It is defined by a stochastic version of the mirror descent algorithm which performs descent of the gradient type in the dual space with an additional averaging. The main result of the paper is an upper bound for the expected accuracy of the proposed estimator. This bound is of the order $$C\sqrt {(\log M)/t}$$ with an explicit and small constant factor C, where M is the dimension of the problem and t stands for the sample size. A similar bound is proved for a more general setting, which covers, in particular, the regression model with squared loss.