Increased accessibility to nonverbal communication through facial and expression recognition technologies for blind/visually impaired subjects

Authors:
Douglas Astler;Harrison Chau;Kailin Hsu;Alvin Hua;Andrew Kannan;Lydia Lei;Melissa Nathanson;Esmaeel Paryavi;Michelle Rosen;Hayato Unno;Carol Wang;Khadija Zaidi;Xuemin Zhang;Cha-Min Tang
Affiliations:
University of Maryland, College Park, MD, USA;University of Maryland, College Park, MD, USA;University of Maryland, College Park, MD, USA;University of Maryland, College Park, MD, USA;University of Maryland, College Park, MD, USA;University of Maryland, College Park, MD, USA;University of Maryland, College Park, MD, USA;University of Maryland, College Park, MD, USA;University of Maryland, College Park, MD, USA;University of Maryland, College Park, MD, USA;University of Maryland, College Park, MD, USA;University of Maryland, College Park, MD, USA;University of Maryland, College Park, MD, USA;University of Maryland School of Medicine, Baltimore, MD, USA
Venue:
The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility
Year:
2011

Citing 2
Cited 2

Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion

International Journal of Computer Vision
A wearable face recognition system for individuals with visual impairments

Proceedings of the 7th international ACM SIGACCESS conference on Computers and accessibility

FEPS: a sensory substitution system for the blind to perceive facial expressions

Proceedings of the 14th international ACM SIGACCESS conference on Computers and accessibility
Do you see what I see?: designing a sensory substitution device to access non-verbal modes of communication

Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility

Quantified Score

Hi-index	0.00

Visualization

Abstract

Conversation between two individuals requires verbal dialogue; the majority of human communication however consists of non-verbal cues such as gestures and facial expressions. Blind individuals are thus hindered in their interaction capabilities. To address this, we are building a computer vision system with facial recognition and expression algorithms to relay nonverbal messages to a blind user. The device will communicate the identities and facial expressions of communication partners in realtime. In order to ensure that this device will be useful to the blind community, we conducted surveys and interviews and we are working with subjects to test prototypes of the device. This paper describes the algorithms and design concepts incorporated in this device, and it provides a commentary on early survey and interview results. A corresponding poster with demonstration stills is exhibited at this conference.