Information complexity of black-box convex optimization: a new look via feedback information theory
Allerton'09 Proceedings of the 47th annual Allerton conference on Communication, control, and computing
IEEE Transactions on Information Theory
Rademacher Complexities and Bounding the Excess Risk in Active Learning
The Journal of Machine Learning Research
Theoretical Computer Science
Smoothness, Disagreement Coefficient, and the Label Complexity of Agnostic Active Learning
The Journal of Machine Learning Research
Unlabeled data and multiple views
PSL'11 Proceedings of the First IAPR TC3 conference on Partially Supervised Learning
Plug-in approach to active learning
The Journal of Machine Learning Research
Activized learning: transforming passive to active with improved label complexity
The Journal of Machine Learning Research
A theory of transfer learning with applications to active learning
Machine Learning
Selective sampling and active learning from single and multiple teachers
The Journal of Machine Learning Research
| Hi-index | 754.90 |
This paper analyzes the potential advantages and theoretical challenges of "active learning" algorithms. Active learning involves sequential sampling procedures that use information gleaned from previous samples in order to focus the sampling and accelerate the learning process relative to "passive learning" algorithms, which are based on nonadaptive (usually random) samples. There are a number of empirical and theoretical results suggesting that in certain situations active learning can be significantly more effective than passive learning. However, the fact that active learning algorithms are feedback systems makes their theoretical analysis very challenging. This paper aims to shed light on achievable limits in active learning. Using minimax analysis techniques, we study the achievable rates of classification error convergence for broad classes of distributions characterized by decision boundary regularity and noise conditions. The results clearly indicate the conditions under which one can expect significant gains through active learning. Furthermore, we show that the learning rates derived are tight for "boundary fragment" classes in d-dimensional feature spaces when the feature marginal density is bounded from above and below.