Learning regular sets from queries and counterexamples
Information and Computation
Letter Recognition Using Holland-Style Adaptive Classifiers
Machine Learning
COLT '92 Proceedings of the fifth annual workshop on Computational learning theory
Bayesian methods for adaptive models
Bayesian methods for adaptive models
Neural networks and the bias/variance dilemma
Neural Computation
A sequential algorithm for training text classifiers
SIGIR '94 Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval
Machine Learning
A maximum entropy approach to natural language processing
Computational Linguistics
Neural network exploration using optimal experiment design
Neural Networks
Selective Sampling Using the Query by Committee Algorithm
Machine Learning
Designing Optimal Sequential Experiments for a Bayesian Classifier
IEEE Transactions on Pattern Analysis and Machine Intelligence
Learning to construct knowledge bases from the World Wide Web
Artificial Intelligence - Special issue on Intelligent internet systems
Machine Learning
Toward Optimal Active Learning through Sampling Estimation of Error Reduction
ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Query Learning Strategies Using Boosting and Bagging
ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
Employing EM and Pool-Based Active Learning for Text Classification
ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
Diverse ensembles for active learning
ICML '04 Proceedings of the twenty-first international conference on Machine learning
Scaling to very very large corpora for natural language disambiguation
ACL '01 Proceedings of the 39th Annual Meeting on Association for Computational Linguistics
Active learning for statistical natural language parsing
ACL '02 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
Example selection for bootstrapping statistical parsers
NAACL '03 Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - Volume 1
Sample Selection for Statistical Parsing
Computational Linguistics
Active learning for logistic regression
Active learning for logistic regression
An empirical study of the behavior of active learning for word sense disambiguation
HLT-NAACL '06 Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics
Active learning for class probability estimation and ranking
IJCAI'01 Proceedings of the 17th international joint conference on Artificial intelligence - Volume 2
Data Mining and Knowledge Discovery
Feature selection with dynamic mutual information
Pattern Recognition
A Density-Based Re-ranking Technique for Active Learning for Data Annotations
ICCPOL '09 Proceedings of the 22nd International Conference on Computer Processing of Oriental Languages. Language Technology for the Knowledge-based Economy
Pool-based active learning in approximate linear regression
Machine Learning
Convex variational Bayesian inference for large scale generalized linear models
ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
On proper unit selection in active learning: co-selection effects for named entity recognition
HLT '09 Proceedings of the NAACL HLT 2009 Workshop on Active Learning for Natural Language Processing
Multi-criteria-based strategy to stop active learning for data annotation
COLING '08 Proceedings of the 22nd International Conference on Computational Linguistics - Volume 1
COLING '08 Proceedings of the 22nd International Conference on Computational Linguistics - Volume 1
Reducing the annotation effort for letter-to-phoneme conversion
ACL '09 Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1 - Volume 1
Confidence-based stopping criteria for active learning for data annotation
ACM Transactions on Speech and Language Processing (TSLP)
Active learning with sampling by uncertainty and density for data annotations
IEEE Transactions on Audio, Speech, and Language Processing
Inactive learning?: difficulties employing active learning in practice
ACM SIGKDD Explorations Newsletter
Handling expensive optimization with large noise
Proceedings of the 11th workshop proceedings on Foundations of genetic algorithms
Evaluating the impact of coder errors on active learning
HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1
Good seed makes a good crop: accelerating active learning using language modeling
HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers - Volume 2
Word clouds for efficient document labeling
DS'11 Proceedings of the 14th international conference on Discovery science
Uncertainty-based active learning with instability estimation for text classification
ACM Transactions on Speech and Language Processing (TSLP)
Active learning of visual descriptors for grasping using non-parametric smoothed beta distributions
Robotics and Autonomous Systems
Active learning with Amazon Mechanical Turk
EMNLP '11 Proceedings of the Conference on Empirical Methods in Natural Language Processing
Activized learning: transforming passive to active with improved label complexity
The Journal of Machine Learning Research
Active learning for coreference resolution
BioNLP '12 Proceedings of the 2012 Workshop on Biomedical Natural Language Processing
LogUCB: an explore-exploit algorithm for comments recommendation
Proceedings of the 21st ACM international conference on Information and knowledge management
Active learning of inverse models with intrinsically motivated goal exploration in robots
Robotics and Autonomous Systems
Efficiently learning the preferences of people
Machine Learning
Autonomously learning to visually detect where manipulation will succeed
Autonomous Robots
| Hi-index | 0.00 |
Which active learning methods can we expect to yield good performance in learning binary and multi-category logistic regression classifiers? Addressing this question is a natural first step in providing robust solutions for active learning across a wide variety of exponential models including maximum entropy, generalized linear, log-linear, and conditional random field models. For the logistic regression model we re-derive the variance reduction method known in experimental design circles as `A-optimality.' We then run comparisons against different variations of the most widely used heuristic schemes: query by committee and uncertainty sampling, to discover which methods work best for different classes of problems and why. We find that among the strategies tested, the experimental design methods are most likely to match or beat a random sample baseline. The heuristic alternatives produced mixed results, with an uncertainty sampling variant called margin sampling and a derivative method called QBB-MM providing the most promising performance at very low computational cost. Computational running times of the experimental design methods were a bottleneck to the evaluations. Meanwhile, evaluation of the heuristic methods lead to an accumulation of negative results. We explore alternative evaluation design parameters to test whether these negative results are merely an artifact of settings where experimental design methods can be applied. The results demonstrate a need for improved active learning methods that will provide reliable performance at a reasonable computational cost.