On learning from queries and counterexamples in the presence of noise
Information Processing Letters
Lower bounds for sampling algorithms for estimating the average
Information Processing Letters
Selective Sampling Using the Query by Committee Algorithm
Machine Learning
Improved lower bounds for learning from noisy examples: an information-theoretic approach
COLT' 98 Proceedings of the eleventh annual conference on Computational learning theory
Machine Learning
Adaptive Sampling Methods for Scaling Up Knowledge Discovery Algorithms
DS '99 Proceedings of the Second International Conference on Discovery Science
Support vector machine active learning with applications to text classification
The Journal of Machine Learning Research
Estimation of Dependences Based on Empirical Data: Springer Series in Statistics (Springer Series in Statistics)
ICML '06 Proceedings of the 23rd international conference on Machine learning
Analysis of perceptron-based active learning
COLT'05 Proceedings of the 18th annual conference on Learning Theory
ICML '06 Proceedings of the 23rd international conference on Machine learning
A bound on the label complexity of agnostic active learning
Proceedings of the 24th international conference on Machine learning
Journal of Computer and System Sciences
Importance weighted active learning
ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Efficient Coverage of Case Space with Active Learning
EPIA '09 Proceedings of the 14th Portuguese Conference on Artificial Intelligence: Progress in Artificial Intelligence
A discriminative model for semi-supervised learning
Journal of the ACM (JACM)
Teaching dimension and the complexity of active learning
COLT'07 Proceedings of the 20th annual conference on Learning theory
Cost-minimising strategies for data labelling: optimal stopping and active learning
FoIKS'08 Proceedings of the 5th international conference on Foundations of information and knowledge systems
d-Confidence: an active learning strategy which efficiently identifies small classes
ALNLP '10 Proceedings of the NAACL HLT 2010 Workshop on Active Learning for Natural Language Processing
Smoothness, Disagreement Coefficient, and the Label Complexity of Agnostic Active Learning
The Journal of Machine Learning Research
The inductive software engineering manifesto: principles for industrial data mining
Proceedings of the International Workshop on Machine Learning Technologies in Software Engineering
Activized learning: transforming passive to active with improved label complexity
The Journal of Machine Learning Research
A theory of transfer learning with applications to active learning
Machine Learning
Hi-index | 0.00 |
Most of the existing active learning algorithms are based on the realizability assumption: The learner's hypothesis class is assumed to contain a target function that perfectly classifies all training and test examples. This assumption can hardly ever be justified in practice. In this paper, we study how relaxing the realizability assumption affects the sample complexity of active learning. First, we extend existing results on query learning to show that any active learning algorithm for the realizable case can be transformed to tolerate random bounded rate class noise. Thus, bounded rate class noise adds little extra complications to active learning, and in particular exponential label complexity savings over passive learning are still possible. However, it is questionable whether this noise model is any more realistic in practice than assuming no noise at all. Our second result shows that if we move to the truly non-realizable model of statistical learning theory, then the label complexity of active learning has the same dependence Ω(1/ε2) on the accuracy parameter ε as the passive learning label complexity. More specifically, we show that under the assumption that the best classifier in the learner's hypothesis class has generalization error at most β0, the label complexity of active learning is Ω(β2/ε2log(1/δ)), where the accuracy parameter ε measures how close to optimal within the hypothesis class the active learner has to get and δ is the confidence parameter. The implication of this lower bound is that exponential savings should not be expected in realistic models of active learning, and thus the label complexity goals in active learning should be refined.