From motion to emotion: a wearable system for the multimedia enrichment of a Butoh dace performance

Authors:
Michael Barry;Juerg Gutknecht;Irena Kulka;Paul Lukowicz;Thomas Stricker
Affiliations:
Institute for Computer Systems & Networks, UMIT, Austria;Institute for Computer Systems & Networks, UMIT, Austria;Hyperwerk, Interaction Design, Univ. of Applied Arts, Switzerland;Electronics Laboratory, Dept. of Electrical Engineering & IT, ETH Zurich, Switzerland;Laboratory for Computer Systems, Dept. of Computer Science, ETH Zurich, Switzerland
Venue:
Journal of Mobile Multimedia
Year:
2005

Citing 13
Cited 3

Computer technologies that support Kansei expression using the body

MULTIMEDIA '01 Proceedings of the ninth ACM international conference on Multimedia
Affect computing in film through sound energy dynamics

MULTIMEDIA '01 Proceedings of the ninth ACM international conference on Multimedia
Toward Machine Emotional Intelligence: Analysis of Affective Physiological State

IEEE Transactions on Pattern Analysis and Machine Intelligence - Graph Algorithms and Computer Vision
The WearARM Modular, Low-Power Computing Core

IEEE Micro
Body-Brush: a body-driven interface for visual aesthetics

Proceedings of the tenth ACM international conference on Multimedia
A Multiprocessor Kernel for Active Object-Based Systems

JMLC '00 Proceedings of the Joint Modular Languages Conference on Modular Programming Languages
Context Awareness by Analyzing Accelerometer Data

ISWC '00 Proceedings of the 4th IEEE International Symposium on Wearable Computers
Affective computing: challenges

International Journal of Human-Computer Studies - Application of affective computing in human—Computer interaction
Wearable sensing to annotate meeting recordings

Personal and Ubiquitous Computing
PadNET: Wearable Physical Activity Detection Network

ISWC '03 Proceedings of the 7th IEEE International Symposium on Wearable Computers
Affective content detection using HMMs

MULTIMEDIA '03 Proceedings of the eleventh ACM international conference on Multimedia
Media in performance: interactive spaces for dance, theater, circus, and museum exhibits

IBM Systems Journal
Interactive systems design: a KANSEI-based approach

NIME '02 Proceedings of the 2002 conference on New interfaces for musical expression

New fair QoS-based charging solution for mobile multimedia streams

International Journal of Virtual Technology and Multimedia
A novel approach for improving the quality of service for wireless video transcoding

Journal of Mobile Multimedia
Human motion recognition using a wireless sensor-based wearable system

Personal and Ubiquitous Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

We present a mobile, multimedia system based on a network of body worn motion sensors, a wearable computer and a visualization engine that is used to produce a visual enhancement of Butoh dance performance. The core of the system is a novel motion classification scheme that allows us to capture the emotion expressed by the dancer during the performance and map it onto scripted visual effects. We describe the artistic concept behind the multimedia enhancement, the motion classification scheme and the system architecture. In an experimental evaluation we investigate the usefulness and the robustness of the wearable computer as well as the classification accuracy of the motion-sensing system. We also summarize the experiences with using the system for live performances on stage in several shows.