Toward a vision-based hand gesture interface
VRST '94 Proceedings of the conference on Virtual reality software and technology
Detection and Recognition of Periodic, Nonrigid Motion
International Journal of Computer Vision
View-Invariant Analysis of Cyclic Motion
International Journal of Computer Vision
IEEE Transactions on Pattern Analysis and Machine Intelligence
Robust Real-Time Periodic Motion Detection, Analysis, and Applications
IEEE Transactions on Pattern Analysis and Machine Intelligence
Multimodal human discourse: gesture and speech
ACM Transactions on Computer-Human Interaction (TOCHI)
Unencumbered Gestural Interaction
IEEE MultiMedia
Vector Coherence Mapping: A Parallelizable Approach to Image Flow Computation
ACCV '98 Proceedings of the Third Asian Conference on Computer Vision-Volume II
Probabilistic Reasoning Models for Face Recognition
CVPR '98 Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
A Parallel Algorithm for Dynamic Gesture Tracking
RATFG-RTS '99 Proceedings of the International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems
Real-time Human Motion Analysis by Image Skeletonization
WACV '98 Proceedings of the 4th IEEE Workshop on Applications of Computer Vision (WACV'98)
The Catchment Feature Model for Multimodal Language Analysis
ICCV '03 Proceedings of the Ninth IEEE International Conference on Computer Vision - Volume 2
Motion-Based Recognition of Pedestrians
ICPR '98 Proceedings of the 14th International Conference on Pattern Recognition-Volume 2 - Volume 2
A robust agent-based gesture tracking system
A robust agent-based gesture tracking system
Multiridge detection and time-frequency reconstruction
IEEE Transactions on Signal Processing
Discourse topic and gestural form
AAAI'08 Proceedings of the 23rd national conference on Artificial intelligence - Volume 2
Gesture salience as a hidden variable for coreference resolution and keyframe extraction
Journal of Artificial Intelligence Research
Automatic multimodal descriptors of rhythmic body movement
Proceedings of the 15th ACM on International conference on multimodal interaction
Hi-index | 0.00 |
Gesture and speech are co-expressive and complementary channels of a single human language system. While speech carries the major load of symbolic presentation, gesture provides the imagistic content. We investigate the role of oscillatory/cyclical hand motions in `carrying' this image content. We present our work on the extraction of hand motion oscillation frequencies of gestures that accompany speech. The key challenges are that such motions are characterized by non-stationary oscillations, and multiple frequencies may be simultaneously extant. Also, the duration of the oscillations may be extended over very few cycles. We apply the windowed Fourier transform and wavelet transform to detect and extract gesticulatory oscillations. We tested these against synthetic signals (stationary and non-stationary) and real data sequences of gesticulatory hand movements in natural discourse. Our results show that both filters functioned well for the synthetic signals. For the real data, the wavelet bandpass filter bank is better for detecting and extracting hand gesture oscillations. We relate the hand motion oscillatory gestures detected by wavelet analysis to speech in natural conversation and apply to multimodal language analysis. We demonstrate the ability of our algorithm to extract gesticulatory oscillations and show how oscillatory gestures reveal portions of the multimodal discourse structure.