Social signals, their function, and automatic analysis: a survey

  • Authors:
  • Alessandro Vinciarelli;Maja Pantic;Hervé Bourlard;Alex Pentland

  • Affiliations:
  • Idiap research institute, Martigny and Ecole Polytechnique Federale de Lausanne, Switzerland;Imperial College London, London, United Kingdom and University of Twente, Enschede, NL;Idiap research institute, Martigny and Ecole Polytechnique Federale de Lausanne, Switzerland;MIT, Boston, USA

  • Venue:
  • ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Social Signal Processing (SSP) aims at the analysis of social behaviour in both Human-Human and Human-Computer interactions. SSP revolves around automatic sensing and interpretation of social signals, complex aggregates of nonverbal behaviours through which individuals express their attitudes towards other human (and virtual) participants in the current social context. As such, SSP integrates both engineering (speech analysis, computer vision, etc.) and human sciences (social psychology, anthropology, etc.) as it requires multimodal and multidisciplinary approaches. As of today, SSP is still in its early infancy, but the domain is quickly developing, and a growing number of works is appearing in the literature. This paper provides an introduction to nonverbal behaviour involved in social signals and a survey of the main results obtained so far in SSP. It also outlines possibilities and challenges that SSP is expected to face in the next years if it is to reach its full maturity.