A sensing architecture for empathetic data systems

Authors:
Johannes Wagner;Florian Lingenfelser;Elisabeth André;Daniele Mazzei;Alessandro Tognetti;Antonio Lanatà;Danilo De Rossi;Alberto Betella;Riccardo Zucca;Pedro Omedas;Paul F. M. J. Verschure
Affiliations:
University of Augsburg Augsburg, Germany;University of Augsburg Augsburg, Germany;University of Augsburg Augsburg, Germany;University of Pisa, Pisa, Italy;University of Pisa, Pisa, Italy;University of Pisa, Pisa, Italy;University of Pisa, Pisa, Italy;Universitat Pompeu Fabra;Universitat Pompeu Fabra;Universitat Pompeu Fabra;Universitat Pompeu Fabra and Institució Catalana de Recerca i Estudis Avançats Barcelona, Spain
Venue:
Proceedings of the 4th Augmented Human International Conference
Year:
2013

Citing 5
Cited 2

E-tree: emotionally driven augmented reality art

MM '08 Proceedings of the 16th ACM international conference on Multimedia
Attentive documents: Eye tracking as implicit feedback for information retrieval and beyond

ACM Transactions on Interactive Intelligent Systems (TiiS)
AutoSelect: What You Want Is What You Get: Real-Time Processing of Visual Attention and Affect

PIT'06 Proceedings of the 2006 international tutorial and research conference on Perception and Interactive Technologies
A wearable health care system based on knitted integrated sensors

IEEE Transactions on Information Technology in Biomedicine
Embodied interaction with complex neuronal data in mixed-reality

Proceedings of the 2012 Virtual Reality International Conference

The social signal interpretation (SSI) framework: multimodal signal processing and recognition in real-time

Proceedings of the 21st ACM international conference on Multimedia
Exploiting unconscious user signals in multimodal human-computer interaction

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP) - Special Sections on the 20th Anniversary of ACM International Conference on Multimedia, Best Papers of ACM Multimedia 2012

Quantified Score

Hi-index	0.00

Visualization

Abstract

Today's increasingly large and complex databases require novel and machine aided ways of exploring data. To optimize the selection and presentation of data, we suggest an unconventional approach. Instead of exclusively relying on explicit user input to specify relevant information or to navigate through a data space, we exploit the power and potential of the users' unconscious processes in addition. To this end, the user is immersed in a mixed reality environment while his bodily reactions are captured using unobtrusive wearable devices. The users' reactions are analyzed in real-time and mapped onto higher-level psychological states, such as surprise or boredom, in order to trigger appropriate system responses that direct the users' attention to areas of potential interest in the visualizations. The realization of such a close experience-based human-machine loop raises a number of technical challenges, such as the real-time interpretation of psychological user states. The paper at hand describes a sensing architecture for empathetic data systems that has been developed as part of such a loop and how it tackles the diverse challenges.