Glove-TalkII: an adaptive gesture-to-formant interface
CHI '95 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
A survey of user-centered design practice
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Proceedings of the 8th international conference on Intelligent user interfaces
XWand: UI for intelligent spaces
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
CueTIP: a mixed-initiative interface for correcting handwriting errors
UIST '06 Proceedings of the 19th annual ACM symposium on User interface software and technology
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
CueFlik: interactive concept learning in image search
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
EnsembleMatrix: interactive visualization to support machine learning with multiple classifiers
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Overview based example selection in end user interactive concept learning
Proceedings of the 22nd annual ACM symposium on User interface software and technology
Interactive optimization for steering machine classification
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Examining multiple potential models in end-user interactive concept learning
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Real-time human interaction with supervised learning algorithms for music composition and performance
A toolkit for designing interactive musical agents
Proceedings of the 23rd Australian Computer-Human Interaction Conference
Tell me more?: the effects of mental model soundness on personalizing an intelligent agent
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
End-user interactions with intelligent and autonomous systems
CHI '12 Extended Abstracts on Human Factors in Computing Systems
The challenges and potential of end-user gesture customization
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
SIG NIME: music, technology, and human-computer interaction
CHI '13 Extended Abstracts on Human Factors in Computing Systems
De-Mo: designing action-sound relationships with the mo interfaces
CHI '13 Extended Abstracts on Human Factors in Computing Systems
Customizing by doing for responsive video game characters
International Journal of Human-Computer Studies
Physical modelling and supervised training of a virtual string quartet
Proceedings of the 21st ACM international conference on Multimedia
A multimodal probabilistic model for gesture--based control of sound synthesis
Proceedings of the 21st ACM international conference on Multimedia
ACM Transactions on Interactive Intelligent Systems (TiiS)
Hi-index | 0.01 |
Model evaluation plays a special role in interactive machine learning (IML) systems in which users rely on their assessment of a model's performance in order to determine how to improve it. A better understanding of what model criteria are important to users can therefore inform the design of user interfaces for model evaluation as well as the choice and design of learning algorithms. We present work studying the evaluation practices of end users interactively building supervised learning systems for real-world gesture analysis problems. We examine users' model evaluation criteria, which span conventionally relevant criteria such as accuracy and cost, as well as novel criteria such as unexpectedness. We observed that users employed evaluation techniques---including cross-validation and direct, real-time evaluation---not only to make relevant judgments of algorithms' performance and interactively improve the trained models, but also to learn to provide more effective training data. Furthermore, we observed that evaluation taught users about what types of models were easy or possible to build, and users sometimes used this information to modify the learning problem definition or their plans for using the trained models in practice. We discuss the implications of these findings with regard to the role of generalization accuracy in IML, the design of new algorithms and interfaces, and the scope of potential benefits of incorporating human interaction in the design of supervised learning systems.