Speech and gestures for graphic image manipulation
CHI '89 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Multimedia and hypertext: the Internet and beyond
Multimedia and hypertext: the Internet and beyond
Survival of the fittest: the evolution of multimedia user interfaces
ACM Computing Surveys (CSUR)
Artificial Intelligence Review - Special issue on integration of natural language and vision processing: recent advances
Affective computing
SUITOR: an attentive information system
Proceedings of the 5th international conference on Intelligent user interfaces
Perceptual user interfaces: perceptual intelligence
Communications of the ACM
Looking at People: Sensing for Ubiquitous and Wearable Computing
IEEE Transactions on Pattern Analysis and Machine Intelligence
Communications of the ACM
Communications of the ACM
IEEE Transactions on Pattern Analysis and Machine Intelligence
Modeling Multimodal Expression of User's Affective Subjective Experience
User Modeling and User-Adapted Interaction
User Modeling and User-Adapted Interaction
What We Talk About When We Talk About Software Agents
IEEE Intelligent Systems
MAUI: a multimodal affective user interface
Proceedings of the tenth ACM international conference on Multimedia
Case-Based Reasoning: Survey and Future Directions
XPS '99 Proceedings of the 5th Biannual German Conference on Knowledge-Based Systems: Knowledge-Based Systems - Survey and Future Directions
“Put-that-there”: Voice and gesture at the graphics interface
SIGGRAPH '80 Proceedings of the 7th annual conference on Computer graphics and interactive techniques
To feel or not to feel: the role of affect in human-computer interaction
International Journal of Human-Computer Studies - Application of affective computing in human—Computer interaction
Communications of the ACM - Multimodal interfaces that flex, adapt, and persist
Multimodal processing by finding common cause
Communications of the ACM - Multimodal interfaces that flex, adapt, and persist
Guidelines for multimodal user interface design
Communications of the ACM - Multimodal interfaces that flex, adapt, and persist
Computer vision in the interface
Communications of the ACM - Multimodal interfaces that flex, adapt, and persist
Challenges in adopting speech recognition
Communications of the ACM - Multimodal interfaces that flex, adapt, and persist
Has the Internet become indispensable?
Communications of the ACM - Has the Internet become indispensable?
Real-Time Inference of Complex Mental States from Facial Expressions and Head Gestures
CVPRW '04 Proceedings of the 2004 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'04) Volume 10 - Volume 10
The importance of affective quality
Communications of the ACM - Special issue: RFID
Affective multimodal human-computer interaction
Proceedings of the 13th annual ACM international conference on Multimedia
Multimodal affect recognition in learning environments
Proceedings of the 13th annual ACM international conference on Multimedia
From brows to trust: evaluating embodied conversational agents
From brows to trust: evaluating embodied conversational agents
Audio-visual emotion recognition in adult attachment interview
Proceedings of the 8th international conference on Multimodal interfaces
Modeling naturalistic affective states via facial and vocal expressions recognition
Proceedings of the 8th international conference on Multimodal interfaces
Gaze-X: adaptive affective multimodal interface for single-user office scenarios
Proceedings of the 8th international conference on Multimodal interfaces
Human computing and machine understanding of human behavior: a survey
Proceedings of the 8th international conference on Multimodal interfaces
Human-Computer Interaction
An automated face reader for fatigue detection
FGR' 04 Proceedings of the Sixth IEEE international conference on Automatic face and gesture recognition
A model based method for automatic facial expression recognition
ECML'05 Proceedings of the 16th European conference on Machine Learning
Multimodal human computer interaction: a survey
ICCV'05 Proceedings of the 2005 international conference on Computer Vision in Human-Computer Interaction
Affective multimodal mirror: sensing and eliciting laughter
Proceedings of the international workshop on Human-centered multimedia
Human-Centred Intelligent Human Computer Interaction (HCI²): how far are we from attaining it?
International Journal of Autonomous and Adaptive Communications Systems
Image and Vision Computing
Guest editorial: special issue on human computing
IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics - Special issue on human computing
A software framework for multimodal human-computer interaction systems
SMC'09 Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics
Gaze quality assisted automatic recognition of social contexts in collaborative Tetris
International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction
Estimating and using absolute and relative viewing distance in interactive systems
Pervasive and Mobile Computing
| Hi-index | 0.00 |
This paper describes an intelligent system that we developed to support affective multimodal human-computer interaction (AMM-HCI) where the user's actions and emotions are modeled and then used to adapt the interaction and support the user in his or her activity. The proposed system, which we named Gaze-X, is based on sensing and interpretation of the human part of the computer's context, known as W5+ (who, where, what, when, why, how). It integrates a number of natural human communicative modalities including speech, eye gaze direction, face and facial expression, and a number of standard HCI modalities like keystrokes, mouse movements, and active software identification, which, in turn, are fed into processes that provide decision making and adapt the HCI to support the user in his or her activity according to his or her preferences. A usability study conducted in an office scenario with a number of users indicates that Gaze-X is perceived as effective, easy to use, useful, and affectively qualitative.