Development of an instrument measuring user satisfaction of the human-computer interface
CHI '88 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
A case for interaction: a study of interactive information retrieval behavior and effectiveness
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Interactive query expansion: a user-based evaluation in a relevance feedback environment
Journal of the American Society for Information Science
The role of transparency in recommender systems
CHI '02 Extended Abstracts on Human Factors in Computing Systems
Evaluating Natural Language Processing Systems: An Analysis and Review
Evaluating Natural Language Processing Systems: An Analysis and Review
User Modeling in Human–Computer Interaction
User Modeling and User-Adapted Interaction
Empirical Evaluation of User Models and User-Adapted Systems
User Modeling and User-Adapted Interaction
The role of context in question answering systems
CHI '03 Extended Abstracts on Human Factors in Computing Systems
Introduction to topic detection and tracking
Topic detection and tracking
Improving Cross-Language Text Retrieval with Human Interactions
HICSS '00 Proceedings of the 33rd Hawaii International Conference on System Sciences-Volume 3 - Volume 3
The concept of relevance in IR
Journal of the American Society for Information Science and Technology
Journal of the American Society for Information Science and Technology
Trust building with explanation interfaces
Proceedings of the 11th international conference on Intelligent user interfaces
Relevance feedback and cross-language information retrieval
Information Processing and Management: an International Journal
Confidence estimation for machine translation
COLING '04 Proceedings of the 20th international conference on Computational Linguistics
User-assisted query translation for interactive cross-language information retrieval
Information Processing and Management: an International Journal
Overview of the CLEF 2005 interactive track
CLEF'05 Proceedings of the 6th international conference on Cross-Language Evalution Forum: accessing Multilingual Information Repositories
Access to recorded interviews: A research agenda
Journal on Computing and Cultural Heritage (JOCCH)
Methods for Evaluating Interactive Information Retrieval Systems with Users
Foundations and Trends in Information Retrieval
Hi-index | 0.00 |
This paper describes a formative evaluation of an integrated multilingual, multimedia information system, a series of user studies designed to guide system development. The system includes automatic speech recognition for English, Chinese, and Arabic, automatic translation from Chinese and Arabic into English, and query-based and profile-based search options. The study design emphasizes repeated evaluation with the same (increasingly experienced) participants, exploration of alternative task designs, rich qualitative and quantitative data collection, and rapid analysis to provide the timely feedback needed to support iterative and responsive development. Results indicate that users presented with materials in a language that they do not know can generate remarkably useful work products, but that integration of transcription, translation, search and profile management poses challenges that would be less evident were each technology to be evaluated in isolation.