Does Context Matter ? - A Quantitative Evaluation in a Real World Maintenance Scenario

  • Authors:
  • Kai Kunze;Florian Wagner;Ersun Kartal;Ernesto Morales Kluge;Paul Lukowicz

  • Affiliations:
  • Embedded Systems Lab, University of Passau, Passau, Germany 94032;Embedded Systems Lab, University of Passau, Passau, Germany 94032;Carl Zeiss AG, Konzernfunktion Forschung und Technologie (KFT-TV), Oberkochen, Germany 73446;Bremer Institut für Produktion und Logistik, University of Bremen, Bremen, Germany 28359;Embedded Systems Lab, University of Passau, Passau, Germany 94032

  • Venue:
  • Pervasive '09 Proceedings of the 7th International Conference on Pervasive Computing
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

We describe a systematic, quantitative study of the benefits using context recognition (specifically task tracking) for a wearable maintenance assistance system. A key objective of the work is to do the evaluation in an environment that is as close as possible to a real world setting. To this end, we use actual maintenance tasks on a complex piece of machinery at an industrial site. Subjects for our study are active Zeiss technicians who have an average of 10 years job experience. In a within subject Wizard of Oz study with the interaction modality as the independent variable we compare three interaction modalities: (1) paper based documentation (2) speech controlled head mounted display (HMD) documentation, and context assisted HMD documentation. The study shows that the paper documentation is 50% and the speech only controlled system 30% slower then context. The statistical significance of 99% and 95% respectively (one sided ANOVA test). We also present results of two questionnaires (custom design and standard NASA TLX) that show a clear majority of subjects considered context to be beneficial in one way or the other. At the same time, the questionnaires reveal a certain level of uneasiness with the new modality.