Multimodal sensing, recognizing and browsing group social dynamics

  • Authors:
  • Zhiwen Yu;Zhiyong Yu;Xingshe Zhou;Yuichi Nakamura

  • Affiliations:
  • School of Computer Science, Northwestern Polytechnical University, Xi'an, Shaanxi, China;School of Computer Science, Northwestern Polytechnical University, Xi'an, Shaanxi, China;School of Computer Science, Northwestern Polytechnical University, Xi'an, Shaanxi, China;Academic Center for Computing and Media Studies, Kyoto University, Kyoto, Japan

  • Venue:
  • Personal and Ubiquitous Computing
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

Group social dynamics is crucial for determining whether a meeting was well organized and the conclusion well reasoned. In this paper, we propose multimodal approaches for sensing, recognition and browsing of social dynamics, specifically human semantic interactions and group interests in small group meetings. Unlike physical interactions (e.g., turn-taking and addressing), the human interactions considered here are incorporated with semantics, i.e., user intention or attitude toward a topic. Group interests are defined as episodes in which participants engaged in an emphatic and heated discussion. We adopt multiple sensors, such as video cameras, microphones and motion sensors for meeting capture. Multimodal methods are proposed for human interaction recognition and group interest recognition based on a variety of features. A graphical user interface, the MMBrowser, is presented for browsing group social dynamics. Experimental results have demonstrated the feasibility of the proposed approaches.