Automatic role recognition in multiparty recordings: using social affiliation networks for feature extraction

Authors:
Hugues Salamin;Sarah Favre;Alessandro Vinciarelli
Affiliations:
Department of Computing Science, University of Glasgow, Glasgow, UK;Idiap Research Institute, Martigny, Switzerland and École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland;Department of Computing Science, University of Glasgow, Glasgow, UK
Venue:
IEEE Transactions on Multimedia
Year:
2009

Citing 15
Cited 9

The Rules Behind Roles: Identifying Speaker Role in Radio Broadcasts

Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence
Pattern Recognition and Machine Learning (Information Science and Statistics)

Pattern Recognition and Machine Learning (Information Science and Statistics)
Sociometry Based Multiparty Audio Recordings Summarization

ICPR '06 Proceedings of the 18th International Conference on Pattern Recognition - Volume 02
Broadcast news story segmentation using social network analysis and hidden markov models

Proceedings of the 15th international conference on Multimedia
Predicting two facets of social verticality in meetings from five-minute time slices and nonverbal cues

ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
Role recognition for meeting participants: an approach based on lexical information and social network analysis

MM '08 Proceedings of the 16th ACM international conference on Multimedia
Predicting the dominant clique in meetings through fusion of nonverbal cues

MM '08 Proceedings of the 16th ACM international conference on Multimedia
Social signal processing: state-of-the-art and future perspectives of an emerging domain

MM '08 Proceedings of the 16th ACM international conference on Multimedia
Initial study on automatic identification of speaker role in broadcast news speech

NAACL-Short '06 Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers
Social signal processing: Survey of an emerging domain

Image and Vision Computing
Automatic nonverbal analysis of social interaction in small groups: A review

Image and Vision Computing
Modeling vocal interaction for text-independent participant characterization in multi-party conversation

SIGdial '08 Proceedings of the 9th SIGdial Workshop on Discourse and Dialogue
RoleNet: movie analysis from the perspective of social networks

IEEE Transactions on Multimedia - Special issue on integration of context and content
The AMI meeting corpus: a pre-announcement

MLMI'05 Proceedings of the Second international conference on Machine Learning for Multimodal Interaction
Speakers Role Recognition in Multiparty Audio Recordings Using Social Network Analysis and Duration Distribution Modeling

IEEE Transactions on Multimedia

Automatic role recognition based on conversational and prosodic behaviour

Proceedings of the international conference on Multimedia
The idiap wolf corpus: exploring group behaviour in a competitive role-playing game

Proceedings of the international conference on Multimedia
Speaker role recognition to help spontaneous conversational speech detection

Proceedings of the 2010 international workshop on Searching spontaneous conversational speech
Improving speech processing trough social signals: automatic speaker segmentation of political debates using role based turn-taking patterns

Proceedings of the 2nd international workshop on Social signal processing
Modelling and analyzing multimodal dyadic interactions using social networks

International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction
Linking speaking and looking behavior patterns with group composition, perception, and performance

Proceedings of the 14th ACM international conference on Multimodal interaction
Human behavior analysis in video surveillance: A Social Signal Processing perspective

Neurocomputing
A gaze-based method for relating group involvement to individual engagement in multimodal multiparty dialogue

Proceedings of the 15th ACM on International conference on multimodal interaction
Characterizing and detecting spontaneous speech: Application to speaker role recognition

Speech Communication

Quantified Score

Hi-index	0.00

Visualization

Abstract

Automatic analysis of social interactions attracts increasing attention in the multimedia community. This letter considers one of the most important aspects of the problem, namely the roles played by individuals interacting in different settings. In particular, this work proposes an automatic approach for the recognition of roles in both production environment contexts (e.g., news and talk-shows) and spontaneous situations (e.g., meetings). The experiments are performed over roughly 90 h of material (one of the largest databases used for role recognition in the literature) and show that the recognition effectiveness depends on how much the roles influence the behavior of people. Furthermore, this work proposes the first approach for modeling mutual dependences between roles and assesses its effect on role recognition performance.