On the contextual analysis of agreement scores

Authors:
Dennis Reidsma;Dirk Heylen;Rieks Op Den Akker
Affiliations:
Human Media Interaction, University of Twente, Enschede, The Netherlands;Human Media Interaction, University of Twente, Enschede, The Netherlands;Human Media Interaction, University of Twente, Enschede, The Netherlands
Venue:
Multimodal corpora
Year:
2009

Citing 6
Cited 3

Reliability measurement without limits

Computational Linguistics
Deducing the visual focus of attention from head pose estimation in dynamic multi-view meeting scenarios

ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
Analyzing disagreements

HumanJudge '08 Proceedings of the Workshop on Human Judgements in Computational Linguistics
Head pose tracking and focus of attention recognition algorithms in meeting rooms

CLEAR'06 Proceedings of the 1st international evaluation conference on Classification of events, activities and relationships
The AMI meeting corpus: a pre-announcement

MLMI'05 Proceedings of the Second international conference on Machine Learning for Multimodal Interaction
A study on visual focus of attention recognition from head pose in a meeting room

MLMI'06 Proceedings of the Third international conference on Machine Learning for Multimodal Interaction

The modulation of cooperation and emotion in dialogue: the REC Corpus

ACLstudent '09 Proceedings of the ACL-IJCNLP 2009 Student Research Workshop
Multimodal corpora annotation: validation methods to assess coding scheme reliability

Multimodal corpora
Classification of feedback expressions in multimodal data

ACLShort '10 Proceedings of the ACL 2010 Conference Short Papers

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper explores the relation between agreement, data quality and machine learning, using the AMI corpus. The paper describes a novel approach that uses contextual information from other modalities to determine a more reliable subset of data, for annotations that have a low overall agreement.