On the evaluation of IR systems
Information Processing and Management: an International Journal - Special issue on evaluation issues in information retrieval
The pragmatics of information retrieval experimentation, revisited
Information Processing and Management: an International Journal - Special issue on evaluation issues in information retrieval
Evaluation measures for interactive information retrieval
Information Processing and Management: an International Journal - Special issue on evaluation issues in information retrieval
Shared workspaces: how do they work and when are they useful?
International Journal of Man-Machine Studies
Overview of the second text retrieval conference (TREC-2)
TREC-2 Proceedings of the second conference on Text retrieval conference
Information seeking in electronic environments
Information seeking in electronic environments
Evaluation of evaluation in information retrieval
SIGIR '95 Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval
ACM Transactions on Computer-Human Interaction (TOCHI)
A case for interaction: a study of interactive information retrieval behavior and effectiveness
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Usability problem identification using both low- and high-fidelity prototypes
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Journal of the American Society for Information Science - Special topic issue on the history of documentation and information science: part II
Measuring usability: are effectiveness, efficiency, and satisfaction really correlated?
Proceedings of the SIGCHI conference on Human Factors in Computing Systems
Formal specification: a roadmap
Proceedings of the Conference on The Future of Software Engineering
A system for supporting cross-lingual information retrieval
Information Processing and Management: an International Journal
Optimizing search by showing results in context
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
The TREC interactive track: an annotated bibliography
Information Processing and Management: an International Journal - Special issue on interactivity at the text retrieval conference (TREC)
Reflections on Mira: interactive evaluation in information retrieval
Journal of the American Society for Information Science
A Practical Guide to Usability Testing
A Practical Guide to Usability Testing
The CLEF 2001 Interactive Track
CLEF '01 Revised Papers from the Second Workshop of the Cross-Language Evaluation Forum on Evaluation of Cross-Language Information Retrieval Systems
CHI '03 Extended Abstracts on Human Factors in Computing Systems
Improving Cross-Language Text Retrieval with Human Interactions
HICSS '00 Proceedings of the 33rd Hawaii International Conference on System Sciences-Volume 3 - Volume 3
Stuff I've seen: a system for personal information retrieval and re-use
Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval
Search strategies in content-based image retrieval
Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval
Using terminological feedback for web search refinement: a log-based study
Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval
Query length in interactive information retrieval
Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval
Fuzzy translation of cross-lingual spelling variants
Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval
Interactive Cross-Language Document Selection
Information Retrieval
A day in the life of web searching: an exploratory study
Information Processing and Management: an International Journal
Display time as implicit feedback: understanding task effects
Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
Journal of the American Society for Information Science and Technology
Mobile search with text messages: designing the user experience for google SMS
CHI '05 Extended Abstracts on Human Factors in Computing Systems
User term feedback in interactive text-based image retrieval
Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
International Journal of Human-Computer Studies
Which user interaction for cross-language information retrieval? Design issues and reflections
Journal of the American Society for Information Science and Technology
Remote conversations: the effects of mediating talk with technology
Human-Computer Interaction
Let's stop pushing the envelope and start addressing it: a reference task agenda for HCI
Human-Computer Interaction
Text or pictures? an eyetracking study of how people view digital video surrogates
CIVR'03 Proceedings of the 2nd international conference on Image and video retrieval
The Turn: Integration of Information Seeking and Retrieval in Context
The Turn: Integration of Information Seeking and Retrieval in Context
Using concept hierarchies in text-based image retrieval: a user evaluation
CLEF'05 Proceedings of the 6th international conference on Cross-Language Evalution Forum: accessing Multilingual Information Repositories
Interactive experiments in object-based retrieval
CIVR'06 Proceedings of the 5th international conference on Image and Video Retrieval
Deriving context from users' evaluations to inform software development
Proceedings of the second international symposium on Information interaction in context
An evaluation framework of user interaction with metadata surrogates
Journal of Information Science
Methods for Evaluating Interactive Information Retrieval Systems with Users
Foundations and Trends in Information Retrieval
Multi Visualization and Dynamic Query for Effective Exploration of Semantic Data
ISWC '09 Proceedings of the 8th International Semantic Web Conference
Proceedings of the third symposium on Information interaction in context
Multidimensional relevance: Prioritized aggregation in a personalized Information Retrieval setting
Information Processing and Management: an International Journal
Evaluation of user comprehension of a novel visual search interface
FDIA'09 Proceedings of the Third BCS-IRSG conference on Future Directions in Information Access
Context-aware retrieval going social
FDIA'09 Proceedings of the Third BCS-IRSG conference on Future Directions in Information Access
Modeling geographic, temporal, and proximity contexts for improving geotemporal search
Journal of the American Society for Information Science and Technology
Evaluating semantic search query approaches with expert and casual users
ISWC'12 Proceedings of the 11th international conference on The Semantic Web - Volume Part II
Hi-index | 0.00 |
This paper discusses the role of user-centred evaluations as an essential method for researching interactive information retrieval. It draws mainly on the work carried out during the Clarity Project where different user-centred evaluations were run during the lifecycle of a cross-language information retrieval system. The iterative testing was not only instrumental to the development of a usable system, but it enhanced our knowledge of the potential, impact, and actual use of cross-language information retrieval technology. Indeed the role of the user evaluation was dual: by testing a specific prototype it was possible to gain a micro-view and assess the effectiveness of each component of the complex system; by cumulating the result of all the evaluations (in total 43 people were involved) it was possible to build a macro-view of how cross-language retrieval would impact on users and their tasks. By showing the richness of results that can be acquired, this paper aims at stimulating researchers into considering user-centred evaluations as a flexible, adaptable and comprehensive technique for investigating non-traditional information access systems.