Communications of the ACM - Special issue on graphical user interfaces
Affective computing
Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review
IEEE Transactions on Pattern Analysis and Machine Intelligence
The visual analysis of human movement: a survey
Computer Vision and Image Understanding
Human motion analysis: a review
Computer Vision and Image Understanding
Perceptual user interfaces (introduction)
Communications of the ACM
Perceptual user interfaces: perceptual intelligence
Communications of the ACM
Perceptual user interfaces: multimodal interfaces that process what comes naturally
Communications of the ACM
Evaluation of eye gaze interaction
Proceedings of the SIGCHI conference on Human Factors in Computing Systems
Automatic Analysis of Facial Expressions: The State of the Art
IEEE Transactions on Pattern Analysis and Machine Intelligence
The Recognition of Human Movement Using Temporal Templates
IEEE Transactions on Pattern Analysis and Machine Intelligence
A survey of computer vision-based human motion capture
Computer Vision and Image Understanding - Modeling people toward vision-based underatanding of a person's shape, appearance, and movement
Detecting Faces in Images: A Survey
IEEE Transactions on Pattern Analysis and Machine Intelligence
Human Activity Recognition Using Multidimensional Indexing
IEEE Transactions on Pattern Analysis and Machine Intelligence
Multimodal 'eyes-free' interaction techniques for wearable devices
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Vision-Based Gesture Recognition: A Review
GW '99 Proceedings of the International Gesture Workshop on Gesture-Based Communication in Human-Computer Interaction
The human-computer interaction handbook
A survey of research on context-aware homes
ACSW Frontiers '03 Proceedings of the Australasian information security workshop conference on ACSW frontiers 2003 - Volume 21
Evaluation of Tracking Methods for Human-Computer Interaction
WACV '02 Proceedings of the Sixth IEEE Workshop on Applications of Computer Vision
Techniques for Interactive Audience Participation
ICMI '02 Proceedings of the 4th IEEE International Conference on Multimodal Interfaces
The production and recognition of emotions in speech: features and algorithms
International Journal of Human-Computer Studies - Application of affective computing in human—Computer interaction
Joint processing of audio-visual information for the recognition of emotional expressions in human-computer interaction
Eye Gaze Estimation from a Single Image of One Eye
ICCV '03 Proceedings of the Ninth IEEE International Conference on Computer Vision - Volume 2
Tracking Articulated Body by Dynamic Markov Network
ICCV '03 Proceedings of the Ninth IEEE International Conference on Computer Vision - Volume 2
Facial expression recognition from video sequences: temporal and static modeling
Computer Vision and Image Understanding - Special issue on Face recognition
Face recognition: A literature survey
ACM Computing Surveys (CSUR)
Perceptual user interfaces using vision-based eye tracking
Proceedings of the 5th international conference on Multimodal interfaces
Tangible multimodal interfaces for safety-critical applications
Communications of the ACM - Multimodal interfaces that flex, adapt, and persist
Guidelines for multimodal user interface design
Communications of the ACM - Multimodal interfaces that flex, adapt, and persist
Computer vision in the interface
Communications of the ACM - Multimodal interfaces that flex, adapt, and persist
Robust clustering of eye movement recordings for quantification of visual interest
Proceedings of the 2004 symposium on Eye tracking research & applications
Experimental evaluation of vision and speech based multimodal interfaces
Proceedings of the 2001 workshop on Perceptive user interfaces
A robust algorithm for reading detection
Proceedings of the 2001 workshop on Perceptive user interfaces
A cognitive meta-analysis of design approaches to interruptions in intelligent environments
CHI '04 Extended Abstracts on Human Factors in Computing Systems
A multimodal learning interface for grounding spoken language in sensory perceptions
ACM Transactions on Applied Perception (TAP)
Ontology and Taxonomy Collaborated Framework for Meeting Classification
ICPR '04 Proceedings of the Pattern Recognition, 17th International Conference on (ICPR'04) Volume 4 - Volume 04
Augmenting and sharing memory with eyeBlog
Proceedings of the the 1st ACM workshop on Continuous archival and retrieval of personal experiences
Bimodal HCI-related affect recognition
Proceedings of the 6th international conference on Multimodal interfaces
BT Technology Journal
IEEE Transactions on Pattern Analysis and Machine Intelligence
Using Eye Region Biometrics to Reveal Affective and Cognitive States
CVPRW '04 Proceedings of the 2004 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'04) Volume 5 - Volume 05
Real-Time Inference of Complex Mental States from Facial Expressions and Head Gestures
CVPRW '04 Proceedings of the 2004 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'04) Volume 10 - Volume 10
Automatic Analysis of Multimodal Group Actions in Meetings
IEEE Transactions on Pattern Analysis and Machine Intelligence
Real-Time Gesture Recognition by Learning and Selective Control of Visual Interest Points
IEEE Transactions on Pattern Analysis and Machine Intelligence
Automatic 2D Hand Tracking in Video Sequences
WACV-MOTION '05 Proceedings of the Seventh IEEE Workshops on Application of Computer Vision (WACV/MOTION'05) - Volume 1 - Volume 01
Conversing with the user based on eye-gaze patterns
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
A living laboratory for the design and evaluation of ubiquitous computing technologies
CHI '05 Extended Abstracts on Human Factors in Computing Systems
Designing, playing, and performing with a vision-based mouth interface
NIME '03 Proceedings of the 2003 conference on New interfaces for musical expression
Real-Time Vision for Human-Computer Interaction
Real-Time Vision for Human-Computer Interaction
Affective multimodal human-computer interaction
Proceedings of the 13th annual ACM international conference on Multimedia
Rules of Play: Game Design Fundamentals
Rules of Play: Game Design Fundamentals
Live Soundscape Composition Based on Synthetic Emotions
IEEE MultiMedia
Gestures over video streams to support remote collaboration on physical tasks
Human-Computer Interaction
Determining driver visual attention with one camera
IEEE Transactions on Intelligent Transportation Systems
Affective video content representation and modeling
IEEE Transactions on Multimedia
A survey on visual surveillance of object motion and behaviors
IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews
Sit straight (and tell me what I did today): a human posture alarm and activity summarization system
CARPE '05 Proceedings of the 2nd ACM workshop on Continuous archival and retrieval of personal experiences
Proceedings of the 11th international conference on Intelligent user interfaces
Human-centered multimedia: representations and challenges
Proceedings of the 1st ACM international workshop on Human-centered multimedia
Multimodal human-computer interaction: A survey
Computer Vision and Image Understanding
HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces
HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces
Gesture Recognition Based on Elastic Deformation Energies
Gesture-Based Human-Computer Interaction and Simulation
Mind and Body: Dialogue and Posture for Affect Detection in Learning Environments
Proceedings of the 2007 conference on Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work
Evaluation Proposal of a Framework for the Integration of Multimodal Interaction in 3D Worlds
Proceedings of the 13th International Conference on Human-Computer Interaction. Part II: Novel Interaction Methods and Techniques
International Journal of Approximate Reasoning
Gaze-X: adaptive, affective, multimodal interface for single-user office scenarios
ICMI'06/IJCAI'07 Proceedings of the ICMI 2006 and IJCAI 2007 international conference on Artifical intelligence for human computing
Evaluating multimodal affective fusion using physiological signals
Proceedings of the 16th international conference on Intelligent user interfaces
Designing of multimodal feedback for enhanced multitasking performance
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Machine Recognition of Music Emotion: A Review
ACM Transactions on Intelligent Systems and Technology (TIST)
Design of human-centric adaptive multimodal interfaces
International Journal of Human-Computer Studies
| Hi-index | 0.00 |
In this paper we review the major approaches to multimodal human computer interaction from a computer vision perspective. In particular, we focus on body, gesture, gaze, and affective interaction (facial expression recognition, and emotion in audio). We discuss user and task modeling, and multimodal fusion, highlighting challenges, open issues, and emerging applications for Multimodal Human Computer Interaction (MMHCI) research.