Conducting a Virtual Orchestra
IEEE MultiMedia
Performance timing for keyframe animation
SCA '04 Proceedings of the 2004 ACM SIGGRAPH/Eurographics symposium on Computer animation
A performance-based technique for timing keyframe animations
Graphical Models
Virtual Orchestra: an immersive computer game for fun and education
Proceedings of the 2006 international conference on Game research and development
Binaural mixing using gestural control interaction
Proceedings of the 5th Audio Mostly Conference: A Conference on Interaction with Sound
iBaton: conducting virtual concerts using smartphones
Proceedings of the 2012 Joint International Conference on Human-Centered Computer Environments
| Hi-index | 0.00 |
A real-time visual recognition system, that enables a human conductor to control an electronic orchestra using gestures of a traditional conductor's baton, is described. The positions of the baton and conductor's hand are identified in a sequence of images from a pair of cameras, and tracked in 3D space. Gestures defining the musical beat are detected in baton's trajectory, and conveyed to a sound synthesis system, as events that control the tempo and the phase of the music. Parameters that can enable the control of the volume of sound are computed from the range variations of the baton and the hand. The system's response is nearly instantaneous. The beat detection is reliable, and precise enough to be used by a professional conductor. It has been used to conduct an integrated electronic performance that combined a synthesized orchestra and animated ballet.