Identifying Major Components of Pictures by Audio Encoding of Colours

Authors:
Guido Bologna;Benoît Deville;Thierry Pun;Michel Vinckenbosch
Affiliations:
Laboratoire d'Informatique Industrielle, University of Applied Science HES-SO, Rue de la Prairie 4, 1202 Geneva, Switzerland;Computer Science Center, University of Geneva, Rue Général Dufour 24, 1211 Geneva, Switzerland;Computer Science Center, University of Geneva, Rue Général Dufour 24, 1211 Geneva, Switzerland;Laboratoire d'Informatique Industrielle, University of Applied Science HES-SO, Rue de la Prairie 4, 1202 Geneva, Switzerland
Venue:
IWINAC '07 Proceedings of the 2nd international work-conference on Nature Inspired Problem-Solving Methods in Knowledge Engineering: Interplay Between Natural and Artificial Computation, Part II
Year:
2007

Citing 1
Cited 3

Eye tracking in coloured image scenes represented by ambisonic fields of musical instrument sounds

IWINAC'05 Proceedings of the First international conference on Mechanisms, Symbols, and Models Underlying Cognition: interplay between natural and artificial computation - Volume Part I

See ColOr: Seeing Colours with an Orchestra

Human Machine Interaction
Blind Navigation along a Sinuous Path by Means of the See ColOr Interface

IWINAC '09 Proceedings of the 3rd International Work-Conference on The Interplay Between Natural and Artificial Computation: Part II: Bioinspired Applications in Artificial and Natural Computation
Color-audio encoding interface for visual substitution: see color matlab-based demo

Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility

Quantified Score

Hi-index	0.00

Visualization

Abstract

The goal of the See ColOr project is to achieve a non-invasive mobility aid for blind users that will use the auditory pathway to represent in real-time frontal image scenes. More particularly, we have developed a prototype which transforms HSL coloured pixels into spatialized classical instrument sounds lasting for 300 ms. Hue is sonified by the timbre of a musical instrument, saturation is one of four possible notes, and luminosity is represented by bass when luminosity is rather dark and singing voice when it is relatively bright. Our first experiments are devoted to static images on the computer screen. Six participants with their eyes covered by a dark tissue were trained to associate colours with musical instruments and then asked to determine on several pictures, objects with specific shapes and colours. In order to simplify the protocol of experiments, we used a tactile tablet, which took the place of the camera. Overall, experiment participants found that colour was helpful for the interpretation of image scenes.