Improving Generalization with Active Learning
Machine Learning - Special issue on structured connectionist systems
Principles of mixed-initiative user interfaces
Proceedings of the SIGCHI conference on Human Factors in Computing Systems
Social translucence: an approach to designing systems that support social processes
ACM Transactions on Computer-Human Interaction (TOCHI) - Special issue on human-computer interaction in the new millennium, Part 1
Machine Learning
Labeling images with a computer game
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
BusyBody: creating and fielding personalized models of the cost of interruption
CSCW '04 Proceedings of the 2004 ACM conference on Computer supported cooperative work
When participants do the capturing: the role of media in diary studies
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Predicting human interruptibility with sensors
ACM Transactions on Computer-Human Interaction (TOCHI)
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
CueTIP: a mixed-initiative interface for correcting handwriting errors
UIST '06 Proceedings of the 19th annual ACM symposium on User interface software and technology
Toward harnessing user feedback for machine learning
Proceedings of the 12th international conference on Intelligent user interfaces
Active Learning with Feedback on Features and Instances
The Journal of Machine Learning Research
Activity sensing in the wild: a field trial of ubifit garden
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Usable artificial intelligence
CHI '08 Extended Abstracts on Human Factors in Computing Systems
Proactive learning: cost-sensitive active learning with multiple imperfect oracles
Proceedings of the 17th ACM conference on Information and knowledge management
Intelligibility and accountability: human considerations in context-aware systems
Human-Computer Interaction
Amplifying community content creation with mixed initiative information extraction
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Corrective feedback and persistent learning for information extraction
Artificial Intelligence
Relevance feedback: a power tool for interactive content-based image retrieval
IEEE Transactions on Circuits and Systems for Video Technology
IUI workshop on interactive machine learning
Proceedings of the companion publication of the 2013 international conference on Intelligent user interfaces companion
Hi-index | 0.00 |
We present two studies that evaluate the accuracy of human responses to an intelligent agent's data classification questions. Prior work has shown that agents can elicit accurate human responses, but the applications vary widely in the data features and prediction information they provide to the labelers when asking for help. In an initial analysis of this work, we found the five most popular features, namely uncertainty, amount and level of context, prediction of an answer, and request for user feedback. We propose that there is a set of these data features and prediction information that maximizes the accuracy of labeler responses. In our first study, we compare accuracy of users of an activity recognizer labeling their own data across the dimensions. In the second study, participants were asked to classify a stranger's emails into folders and strangers' work activities by interruptibility. We compared the accuracy of the responses to the users' self-reports across the same five dimensions. We found very similar combinations of information (for users and strangers) that led to very accurate responses as well as more feedback that the agents could use to refine their predictions. We use these results for insight into the information that help labelers the most.