SCANMail: a voicemail interface that makes speech browsable, readable and searchable
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
The TIPSTER SUMMAC Text Summarization Evaluation
EACL '99 Proceedings of the ninth conference on European chapter of the Association for Computational Linguistics
Deep Read: a reading comprehension system
ACL '99 Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics
A meeting browser evaluation test
CHI '05 Extended Abstracts on Human Factors in Computing Systems
Design and evaluation of systems to support interaction capture and retrieval
Personal and Ubiquitous Computing - Special Issue: User-centred design and evaluation of ubiquitous groupware
Combining multiple information layers for the automatic generation of indicative meeting abstracts
ENLG '07 Proceedings of the Eleventh European Workshop on Natural Language Generation
Term-weighting for summarization of multi-party spoken dialogues
MLMI'07 Proceedings of the 4th international conference on Machine learning for multimodal interaction
The AMI meeting corpus: a pre-announcement
MLMI'05 Proceedings of the Second international conference on Machine Learning for Multimodal Interaction
Accessing multimodal meeting data: systems, problems and possibilities
MLMI'04 Proceedings of the First international conference on Machine Learning for Multimodal Interaction
Browsing recorded meetings with ferret
MLMI'04 Proceedings of the First international conference on Machine Learning for Multimodal Interaction
Interpretation and transformation for abstracting conversations
HLT '10 Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics
Abstractive summarization of voice communications
LTC'09 Proceedings of the 4th conference on Human language technology: challenges for computer science and linguistics
Visual structured summaries of human conversations
Proceedings of the first international workshop on Intelligent visual interfaces for text analysis
Participants' personal note-taking in meetings and its value for automatic meeting summarisation
Information Technology and Management
An ecologically valid evaluation of speech summarization
CHI '12 Extended Abstracts on Human Factors in Computing Systems
Ecological validity and the evaluation of speech summarization quality
Proceedings of Workshop on Evaluation Metrics and System Comparison for Automatic Summarization
| Hi-index | 0.00 |
In this work we describe a large-scale extrinsic evaluation of automatic speech summarization technologies for meeting speech. The particular task is a decision audit, wherein a user must satisfy a complex information need, navigating several meetings in order to gain an understanding of how and why a given decision was made. We compare the usefulness of extractive and abstractive technologies in satisfying this information need, and assess the impact of automatic speech recognition (ASR) errors on user performance. We employ several evaluation methods for participant performance, including post-questionnaire data, human subjective and objective judgments, and an analysis of participant browsing behaviour.