Do you see what I see?: designing a sensory substitution device to access non-verbal modes of communication

Authors:
M. Iftekhar Tanveer;A. S. M. Iftekhar Anam;Mohammed Yeasin;Majid Khan
Affiliations:
University of Maryland, Baltimore County, Baltimore, MD;University of Memphis, Memphis, TN;University of Memphis, Memphis, TN;Clovernook Center for the Blind and Visually, Impaired, Memphis TN
Venue:
Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility
Year:
2013

Citing 9
Cited 0

Contextual Design: Defining Customer-Centered Systems

Contextual Design: Defining Customer-Centered Systems
Sonification of Spatial Information: Audio-Tactile Exploration Strategies by Normal and Blind Subjects

UAHCI '09 Proceedings of the 5th International on ConferenceUniversal Access in Human-Computer Interaction. Part II: Intelligent and Ubiquitous Interaction Environments
Freedom to roam: a study of mobile device adoption and accessibility for people with visual and motor disabilities

Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility
Deformable Model Fitting by Regularized Landmark Mean-Shift

International Journal of Computer Vision
Increased accessibility to nonverbal communication through facial and expression recognition technologies for blind/visually impaired subjects

The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility
A spatio-temporal probabilistic framework for dividing and predicting facial action units

ACII'11 Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part II
An FFT-based technique for translation, rotation, and scale-invariant image registration

IEEE Transactions on Image Processing
Learning non-visual graphical information using a touch-based vibro-audio interface

Proceedings of the 14th international ACM SIGACCESS conference on Computers and accessibility
FEPS: a sensory substitution system for the blind to perceive facial expressions

Proceedings of the 14th international ACM SIGACCESS conference on Computers and accessibility

Quantified Score

Hi-index	0.00

Visualization

Abstract

The inability to access non-verbal cues is a setback for people who are blind or visually impaired. A visual-to-auditory Sensory Substitution Device (SSD) may help improve the quality of their lives by transforming visual cues into auditory cues. In this paper, we describe the design and development of a robust and real-time SSD called iFEPS -- improved Facial Expression Perception through Sound. The implementation of the iFEPS evolved over time through a participatory design process. We conducted both subjective and objective experiments to quantify the usability of the system. Evaluation with 14 subjects (7 blind + 7 blind-folded) shows that the users were able to perceive the facial expressions in most of the time. In addition, the overall subjective usability of the system was found to be scoring 4.02 in a 5 point Likert scale.