Fast training of support vector machines using sequential minimal optimization
Advances in kernel methods
Large Margin Classification Using the Perceptron Algorithm
Machine Learning - The Eleventh Annual Conference on computational Learning Theory
A Database for Handwritten Text Recognition Research
IEEE Transactions on Pattern Analysis and Machine Intelligence
Sparse on-line Gaussian processes
Neural Computation
Discriminative Reranking for Natural Language Parsing
ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
A new approximate maximal margin classification algorithm
The Journal of Machine Learning Research
Exact simplification of support vector solutions
The Journal of Machine Learning Research
Ultraconservative online algorithms for multiclass problems
The Journal of Machine Learning Research
The Robustness of the p-Norm Algorithms
Machine Learning
Support vector machine learning for interdependent and structured output spaces
ICML '04 Proceedings of the twenty-first international conference on Machine learning
Vehicle classification in distributed sensor networks
Journal of Parallel and Distributed Computing
A Second-Order Perceptron Algorithm
SIAM Journal on Computing
A Modified Finite Newton Method for Fast Solution of Large Scale Linear SVMs
The Journal of Machine Learning Research
Learning Theory: An Approximation Theory Viewpoint (Cambridge Monographs on Applied & Computational Mathematics)
Online Passive-Aggressive Algorithms
The Journal of Machine Learning Research
The Forgetron: A Kernel-Based Perceptron on a Budget
SIAM Journal on Computing
Tracking the best hyperplane with a simple budget perceptron
COLT'06 Proceedings of the 19th annual conference on Learning Theory
IEEE Transactions on Signal Processing
The kernel recursive least-squares algorithm
IEEE Transactions on Signal Processing
On the generalization ability of on-line learning algorithms
IEEE Transactions on Information Theory
Kernel slicing: scalable online training with conjunctive features
COLING '10 Proceedings of the 23rd International Conference on Computational Linguistics
Example-dependent basis vector selection for kernel-based classifiers
ECML PKDD'10 Proceedings of the 2010 European conference on Machine learning and knowledge discovery in databases: Part III
A kernel-based Perceptron with dynamic memory
Neural Networks
Information Sciences: an International Journal
BDUOL: double updating online learning on a fixed budget
ECML PKDD'12 Proceedings of the 2012 European conference on Machine Learning and Knowledge Discovery in Databases - Volume Part I
Online learning with multiple kernels: A review
Neural Computation
The Journal of Machine Learning Research
Kernel minimum error entropy algorithm
Neurocomputing
Large scale online kernel classification
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Hi-index | 0.00 |
A common problem of kernel-based online algorithms, such as the kernel-based Perceptron algorithm, is the amount of memory required to store the online hypothesis, which may increase without bound as the algorithm progresses. Furthermore, the computational load of such algorithms grows linearly with the amount of memory used to store the hypothesis. To attack these problems, most previous work has focused on discarding some of the instances, in order to keep the memory bounded. In this paper we present a new algorithm, in which the instances are not discarded, but are instead projected onto the space spanned by the previous online hypothesis. We call this algorithm Projectron. While the memory size of the Projectron solution cannot be predicted before training, we prove that its solution is guaranteed to be bounded. We derive a relative mistake bound for the proposed algorithm, and deduce from it a slightly different algorithm which outperforms the Perceptron. We call this second algorithm Projectron++. We show that this algorithm can be extended to handle the multiclass and the structured output settings, resulting, as far as we know, in the first online bounded algorithm that can learn complex classification tasks. The method of bounding the hypothesis representation can be applied to any conservative online algorithm and to other online algorithms, as it is demonstrated for ALMA2. Experimental results on various data sets show the empirical advantage of our technique compared to various bounded online algorithms, both in terms of memory and accuracy.