The weighted majority algorithm
Information and Computation
General convergence results for linear discriminant updates
COLT '97 Proceedings of the tenth annual conference on Computational learning theory
Machine Learning - Special issue on context sensitivity and concept drift
Machine Learning - Special issue on context sensitivity and concept drift
Large Margin Classification Using the Perceptron Algorithm
Machine Learning - The Eleventh Annual Conference on computational Learning Theory
Linear hinge loss and average margin
Proceedings of the 1998 conference on Advances in neural information processing systems II
The Relaxed Online Maximum Margin Algorithm
Machine Learning
Machine Learning
Machine Learning
Tracking the best linear predictor
The Journal of Machine Learning Research
A new approximate maximal margin classification algorithm
The Journal of Machine Learning Research
The Robustness of the p-Norm Algorithms
Machine Learning
A Second-Order Perceptron Algorithm
SIAM Journal on Computing
IEEE Transactions on Signal Processing
Online Regression Competitive with Changing Predictors
ALT '07 Proceedings of the 18th international conference on Algorithmic Learning Theory
Robust bounds for classification via selective sampling
ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
A simpler unified analysis of budget perceptrons
ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
λ-Perceptron: An adaptive classifier for data streams
Pattern Recognition
Kernel slicing: scalable online training with conjunctive features
COLING '10 Proceedings of the 23rd International Conference on Computational Linguistics
Double Updating Online Learning
The Journal of Machine Learning Research
Re-adapting the regularization of weights for non-stationary regression
ALT'11 Proceedings of the 22nd international conference on Algorithmic learning theory
Online feature selection for mining big data
Proceedings of the 1st International Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and Applications
A kernel fused perceptron for the online classification of large-scale data
Proceedings of the 1st International Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and Applications
BDUOL: double updating online learning on a fixed budget
ECML PKDD'12 Proceedings of the 2012 European conference on Machine Learning and Knowledge Discovery in Databases - Volume Part I
New analysis and algorithm for learning with drifting distributions
ALT'12 Proceedings of the 23rd international conference on Algorithmic Learning Theory
Online Multiple Kernel Classification
Machine Learning
Online learning with multiple kernels: A review
Neural Computation
Large scale online kernel classification
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
| Hi-index | 0.00 |
Shifting bounds for on-line classification algorithms ensure good performance on any sequence of examples that is well predicted by a sequence of changing classifiers. When proving shifting bounds for kernel-based classifiers, one also faces the problem of storing a number of support vectors that can grow unboundedly, unless an eviction policy is used to keep this number under control. In this paper, we show that shifting and on-line learning on a budget can be combined surprisingly well. First, we introduce and analyze a shifting Perceptron algorithm achieving the best known shifting bounds while using an unlimited budget. Second, we show that by applying to the Perceptron algorithm the simplest possible eviction policy, which discards a random support vector each time a new one comes in, we achieve a shifting bound close to the one we obtained with no budget restrictions. More importantly, we show that our randomized algorithm strikes the optimal trade-off $$U = \Theta(\sqrt{B})$$ between budget B and norm U of the largest classifier in the comparison sequence. Experiments are presented comparing several linear-threshold algorithms on chronologically-ordered textual datasets. These experiments support our theoretical findings in that they show to what extent randomized budget algorithms are more robust than deterministic ones when learning shifting target data streams.