Random generation of combinatorial structures from a uniform
Theoretical Computer Science
From on-line to batch learning
COLT '89 Proceedings of the second annual workshop on Computational learning theory
The perception: a probabilistic model for information storage and organization in the brain
Neurocomputing: foundations of research
Redundant noisy attributes, attribute errors, and linear-threshold learning using winnow
COLT '91 Proceedings of the fourth annual workshop on Computational learning theory
C4.5: programs for machine learning
C4.5: programs for machine learning
On learning embedded symmetric concepts
COLT '93 Proceedings of the sixth annual conference on Computational learning theory
The weighted majority algorithm
Information and Computation
Weakly learning DNF and characterizing statistical query learning using Fourier analysis
STOC '94 Proceedings of the twenty-sixth annual ACM symposium on Theory of computing
The Markov chain Monte Carlo method: an approach to approximate counting and integration
Approximation algorithms for NP-hard problems
Solving the multiple instance problem with axis-parallel rectangles
Artificial Intelligence
Simple learning algorithms using divide and conquer
Computational Complexity
Predicting Nearly As Well As the Best Pruning of a Decision Tree
Machine Learning - Special issue on the eighth annual conference on computational learning theory, (COLT '95)
Journal of the ACM (JACM)
A decision-theoretic generalization of on-line learning and an application to boosting
Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
Artificial Intelligence - Special issue on relevance
Efficient learning with virtual threshold gates
Information and Computation
Improved Boosting Algorithms Using Confidence-rated Predictions
Machine Learning - The Eleventh Annual Conference on computational Learning Theory
Linear hinge loss and average margin
Proceedings of the 1998 conference on Advances in neural information processing systems II
An introduction to support Vector Machines: and other kernel-based learning methods
An introduction to support Vector Machines: and other kernel-based learning methods
Agnostic learning of geometric patterns
Journal of Computer and System Sciences
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Predicting nearly as well as the best pruning of a planar decision graph
Theoretical Computer Science
ICML '97 Proceedings of the Fourteenth International Conference on Machine Learning
Random Walks on Truncated Cubes and Sampling 0-1 Knapsack Solutions
FOCS '99 Proceedings of the 40th Annual Symposium on Foundations of Computer Science
Multiple-Instance Learning of Real-Valued Geometric Patterns
Annals of Mathematics and Artificial Intelligence
More efficient PAC-learning of DNF with membership queries under the uniform distribution
Journal of Computer and System Sciences
AAAI'96 Proceedings of the thirteenth national conference on Artificial intelligence - Volume 1
An introduction to kernel-based learning algorithms
IEEE Transactions on Neural Networks
Online Rule Learning via Weighted Model Counting
Proceedings of the 2008 conference on ECAI 2008: 18th European Conference on Artificial Intelligence
Efficiency versus convergence of Boolean kernels for on-line learning algorithms
Journal of Artificial Intelligence Research
| Hi-index | 0.00 |
Multiplicative weight-update algorithms such as Winnow and Weighted Majority have been studied extensively due to their on-line mistake bounds' logarithmic dependence on N, the total number of inputs, which allows them to be applied to problems where N is exponential. However, a large N requires techniques to efficiently compute the weighted sums of inputs to these algorithms. In special cases, the weighted sum can be exactly computed efficiently, but for numerous problems such an approach seems infeasible. Thus we explore applications of Markov chain Monte Carlo (MCMC) methods to estimate the total weight. Our methods are very general and applicable to any representation of a learning problem for which the inputs to a linear learning algorithm can be represented as states in a completely connected, untruncated Markov chain. We give theoretical worst-case guarantees on our technique and then apply it to two problems: learning DNF formulas using Winnow, and pruning classifier ensembles using Weighted Majority. We then present empirical results on simulated data indicating that in practice, the time complexity is much better than what is implied by our worst-case theoretical analysis.