BodyExplorerAR: enhancing a mannequin medical simulator with sensing and projective augmented reality for exploring dynamic anatomy and physiology

Authors:
Joseph T. Samosky;Douglas A. Nelson;Bo Wang;Russell Bregman;Andrew Hosmer;Brandon Mikulis;Robert Weaver
Affiliations:
University of Pittsburgh, Pittsburgh, PA;University of Pittsburgh, Pittsburgh, PA;University of Pittsburgh, Pittsburgh, PA;University of Pittsburgh, Pittsburgh, PA;University of Pittsburgh, Pittsburgh, PA;University of Pittsburgh, Pittsburgh, PA;University of Pittsburgh, Pittsburgh, PA
Venue:
Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction
Year:
2012

Citing 9
Cited 0

Interacting with paper on the DigitalDesk

Communications of the ACM - Special issue on computer augmented environments: back to the real world
The office of the future: a unified approach to image-based modeling and spatially immersive displays

Proceedings of the 25th annual conference on Computer graphics and interactive techniques
Spatially augmented reality

IWAR '98 Proceedings of the international workshop on Augmented reality : placing artificial objects in real scenes: placing artificial objects in real scenes
A Flexible Tracking Concept Applied to Medical Scenarios Using an AR Window

ISMAR '02 Proceedings of the 1st International Symposium on Mixed and Augmented Reality
Dynamic Shader Lamps: Painting on Movable Objects

ISAR '01 Proceedings of the IEEE and ACM International Symposium on Augmented Reality (ISAR'01)
Hacking the Nintendo Wii Remote

IEEE Pervasive Computing
Contextual Anatomic Mimesis Hybrid In-Situ Visualization Method for Improving Multi-Sensory Depth Perception in Medical Augmented Reality

ISMAR '07 Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality
Using a depth camera as a touch sensor

ACM International Conference on Interactive Tabletops and Surfaces
AnatOnMe: facilitating doctor-patient communication using a projection-based handheld device

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

BodyExplorerAR is a system designed to enhance a learner's ability to explore anatomy, physiology and clinical interventions though naturalistic interaction with an augmented reality enhanced full-body mannequin simulator. We are developing a platform that integrates projective AR and multi-modal sensor inputs. A user can use an IR pen to open, resize and move viewports providing windows into the body that can display dynamic anatomy. The user can point to an organ and display additional information such as graphs of physiological parameters or heart sounds. Custom sensing systems provide natural interactions with common medical devices such as syringes, breathing tubes and catheters. A user can open a window displaying the beating heart in situ, display an electrocardiogram (ECG), then inject drugs and see and hear changes in heart rate. Our goal is an engaging experience that empowers a learner to create customized, media-rich explorations revealing the internal consequences of external actions.