Mistake bounds and logarithmic linear-threshold learning algorithms
Mistake bounds and logarithmic linear-threshold learning algorithms
A training algorithm for optimal margin classifiers
COLT '92 Proceedings of the fifth annual workshop on Computational learning theory
Robust trainability of single neurons
Journal of Computer and System Sciences
Exponentiated gradient versus gradient descent for linear predictors
Information and Computation
Artificial Intelligence - Special issue on relevance
The robustness of the p-norm algorithms
COLT '99 Proceedings of the twelfth annual conference on Computational learning theory
Large Margin Classification Using the Perceptron Algorithm
Machine Learning - The Eleventh Annual Conference on computational Learning Theory
Linear hinge loss and average margin
Proceedings of the 1998 conference on Advances in neural information processing systems II
Relative Loss Bounds for Multidimensional Regression Problems
Machine Learning
AI Game Programming Wisdom
General Convergence Results for Linear Discriminant Updates
Machine Learning
Large Margin Classification for Moving Targets
ALT '02 Proceedings of the 13th International Conference on Algorithmic Learning Theory
Maximizing the Margin with Boosting
COLT '02 Proceedings of the 15th Annual Conference on Computational Learning Theory
Tracking Linear-Threshold Concepts with Winnow
COLT '02 Proceedings of the 15th Annual Conference on Computational Learning Theory
Tracking the best linear predictor
The Journal of Machine Learning Research
A new approximate maximal margin classification algorithm
The Journal of Machine Learning Research
Estimation of Dependences Based on Empirical Data: Springer Series in Statistics (Springer Series in Statistics)
Relative loss bounds for single neurons
IEEE Transactions on Neural Networks
Machine learning: a review of classification and combining techniques
Artificial Intelligence Review
Boosting expert ensembles for rapid concept recall
AAAI'06 Proceedings of the 21st national conference on Artificial intelligence - Volume 1
Two one-pass algorithms for data stream classification using approximate MEBs
ICANNGA'11 Proceedings of the 10th international conference on Adaptive and natural computing algorithms - Volume Part II
| Hi-index | 0.00 |
This paper surveys some basic techniques and recent results related to online learning. Our focus is on linear classification. The most familiar algorithm for this task is the perceptron. We explain the perceptron algorithm and its convergence proof as an instance of a generic method based on Bregman divergences. This leads to a more general algorithm known as the p-norm perceptron. We give the proof for generalizing the perceptron convergence theorem for the p-norm perceptron and the non-separable case. We also show how regularization, again based on Bregman divergences, can make an online algorithm more robust against target movement.