SOAR: an architecture for general intelligence
Artificial Intelligence
Instance-Based Learning Algorithms
Machine Learning
Learning with an unreliable teacher
Pattern Recognition
Bounds on the mean classification error rate of multiple experts
Pattern Recognition Letters
The Architecture of Cognition
Generating Accurate Rule Sets Without Global Optimization
ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
Active Sampling for Class Probability Estimation and Ranking
Machine Learning
Get another label? improving data quality and data mining using multiple, noisy labelers
Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining
Proceedings of the ACM SIGKDD Workshop on Human Computation
Proceedings of the ACM SIGKDD Workshop on Human Computation
| Hi-index | 0.00 |
Active learning methods have been proposed to reduce the labeling effort of human experts: based on the initially available labeled instances and information about the unlabeled data those algorithms choose only the most informative instances for labeling. They have been shown to significantly reduce the size of the required labeled dataset to generate a precise model [17]. However, active learning framework assumes "perfect" labelers, which is not true in practice (e.g., [22, 23]). In particular, an empirical study for hand-written digit recognition [5] has shown that active learning works poorly when a human labeler is used. Thus, as active learning enters the realm of practical applications, it will need to confront the practicalities and inaccuracies of human expert decision-making. Specifically, active learning approaches will have to deal with the problem that human experts are likely to make mistakes when labeling the selected instances.