Misuse of automated decision aids: Complacency, automation bias and the impact of training experience

  • Authors:

  • J. Elin Bahner;Anke-Dorothea Hüper;Dietrich Manzey

  • Affiliations:

  • Berlin Institute of Technology, Department of Psychology and Ergonomics, Marchstrasse 12, Sekr. F7, 10587 Berlin, Germany;Berlin Institute of Technology, Department of Psychology and Ergonomics, Marchstrasse 12, Sekr. F7, 10587 Berlin, Germany;Berlin Institute of Technology, Department of Psychology and Ergonomics, Marchstrasse 12, Sekr. F7, 10587 Berlin, Germany

  • Venue:
  • International Journal of Human-Computer Studies
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

The present study investigates automation misuse based on complacency and automation bias in interacting with a decision aid in a process control system. The effect of a preventive training intervention which includes exposing participants to rare automation failures is examined. Complacency is reflected in an inappropriate checking and monitoring of automated functions. In interaction with automated decision aids complacency might result in commission errors, i.e., following automatically generated recommendations even though they are false. Yet, empirical evidence proving this kind of relationship is still lacking. A laboratory experiment (N=24) was conducted using a process control simulation. An automated decision aid provided advice for fault diagnosis and management. Complacency was directly measured by the participants' information sampling behavior, i.e., the amount of information sampled in order to verify the automated recommendations. Possible commission errors were assessed when the aid provided false recommendations. The results provide clear evidence for complacency, reflected in an insufficient verification of the automation, while commission errors were associated with high levels of complacency. Hence, commission errors seem to be a possible, albeit not an inevitable consequence of complacency. Furthermore, exposing operators to automation failures during training significantly decreased complacency and thus represents a suitable means to reduce this risk, even though it might not avoid it completely. Potential applications of this research include the design of training protocols in order to prevent automation misuse in interaction with automated decision aids.