Performance of optical flow techniques
International Journal of Computer Vision
The computation of optical flow
ACM Computing Surveys (CSUR)
An isometric tongue pointing device
Proceedings of the ACM SIGCHI Conference on Human factors in computing systems
Facing the music: a facial action controlled musical interface
CHI '01 Extended Abstracts on Human Factors in Computing Systems
View-Based Interpretation of Real-Time Optical Flow for Gesture Recognition
FG '98 Proceedings of the 3rd. International Conference on Face & Gesture Recognition
Enhancing Web-Based Learning by Sharing Affective Experience
ICALT '04 Proceedings of the IEEE International Conference on Advanced Learning Technologies
Tongue 'n' Groove: an ultrasound based music controller
NIME '02 Proceedings of the 2002 conference on New interfaces for musical expression
Interactive Gesture Music performance interface
NIME '02 Proceedings of the 2002 conference on New interfaces for musical expression
Designing, playing, and performing with a vision-based mouth interface
NIME '03 Proceedings of the 2003 conference on New interfaces for musical expression
Contexts of collaborative musical experiences
NIME '03 Proceedings of the 2003 conference on New interfaces for musical expression
Head-tracking for gestural and continuous control of parameterized audio effects
NIME '03 Proceedings of the 2003 conference on New interfaces for musical expression
EyeMusic: making music with the eyes
NIME '04 Proceedings of the 2004 conference on New interfaces for musical expression
Ambient Display using Musical Effects
Proceedings of the 11th international conference on Intelligent user interfaces
An interactive character animation system for dementia care
ACM SIGGRAPH 2006 Research posters
iMime: an interactive character animation system for use in dementia care
Proceedings of the 12th international conference on Intelligent user interfaces
Facial expression recognition as a creative interface
Proceedings of the 13th international conference on Intelligent user interfaces
Creating new interfaces for musical expression: introduction to NIME
ACM SIGGRAPH 2009 Courses
Sonify your face: facial expressions for sound generation
Proceedings of the international conference on Multimedia
Multimedia Tools and Applications
Advances in new interfaces for musical expression
ACM SIGGRAPH 2011 Courses
Advances in new interfaces for musical expression
SIGGRAPH Asia 2012 Courses
Creating new interfaces for musical expression
SIGGRAPH Asia 2013 Courses
| Hi-index | 0.00 |
The central role of the face in social interaction and non-verbal communication suggest we explore facial action as a means of musical expression. This paper presents the design, implementation, and preliminary studies of a novel system utilizing face detection and optic flow algorithms to associate facial movements with sound synthesis in a topographically specific fashion. We report on our experience with various gesture-to-sound mappings and applications, and describe our preliminary experiments at musical performance using the system.