Exploring emotions and multimodality in digitally augmented puppeteering

Authors:
Lassi A. Liikkanen;Giulio Jacucci;Eero Huvio;Toni Laitinen;Elisabeth Andre
Affiliations:
Helsinki Institute for Information Technology HIIT, TKK, Finland;Helsinki Institute for Information Technology HIIT, TKK, Finland;Helsinki Institute for Information Technology HIIT, TKK, Finland;Helsinki Institute for Information Technology HIIT, TKK, Finland;University of Augsburg, Augsburg, Germany
Venue:
AVI '08 Proceedings of the working conference on Advanced visual interfaces
Year:
2008

Citing 14
Cited 1

HoloWall: designing a finger, hand, body, and object sensitive wall

Proceedings of the 10th annual ACM symposium on User interface software and technology
Creating Emotion in Games: The Art and Craft of Emotioneering

Creating Emotion in Games: The Art and Craft of Emotioneering
Towards tangibility in gameplay: building a tangible affective interface for a computer game

Proceedings of the 5th international conference on Multimodal interfaces
VisionWand: interaction techniques for large displays using a passive wand tracked in 3D

Proceedings of the 16th annual ACM symposium on User interface software and technology
Multimodal Acting in Mixed Reality Interactive Storytelling

IEEE MultiMedia
Communicating Expressiveness and Affect in Multimodal Interactive Systems

IEEE MultiMedia
Using frustration in the design of adaptive videogames

Proceedings of the 2004 ACM SIGCHI International Conference on Advances in computer entertainment technology
Empathic painting: interactive stylization through observed emotional state

Proceedings of the 4th international symposium on Non-photorealistic animation and rendering
Human Physiology as a Basis for Designing and Evaluating Affective Communication with Life-Like Characters

IEICE - Transactions on Information and Systems
Modeling and evaluating empathy in embodied companion agents

International Journal of Human-Computer Studies
Editorial: Evaluating affective interactions

International Journal of Human-Computer Studies
Appropriation of a MMS-based comic creator: from system functionalities to resources for action

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
It's Mine, Don't Touch!: interactions at a large multi-touch display in a city centre

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
AI-mediated interaction in virtual reality art

INTETAIN'05 Proceedings of the First international conference on Intelligent Technologies for Interactive Entertainment

Engaging spect-actors with multimodal digital puppetry

Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries

Quantified Score

Hi-index	0.00

Visualization

Abstract

Recently, multimodal and affective technologies have been adopted to support expressive and engaging interaction, bringing up a plethora of new research questions. Among the challenges, two essential topics are 1) how to devise truly multimodal systems that can be used seamlessly for customized performance and content generation, and 2) how to utilize the tracking of emotional cues and respond to them in order to create affective interaction loops. We present PuppetWall, a multi-user, multimodal system intended for digitally augmented puppeteering. This application allows natural interaction to control puppets and manipulate playgrounds comprising background, props, and puppets. PuppetWall utilizes hand movement tracking, a multi-touch display and emotion speech recognition input for interfacing. Here we document the technical features of the system and an initial evaluation. The evaluation involved two professional actors and also aimed at exploring naturally emerging expressive speech categories. We conclude by summarizing challenges in tracking emotional cues from acoustic features and their relevance for the design of affective interactive systems.