Experimental comparison of attack trees and misuse cases for security threat identification

  • Authors:

  • Andreas L. Opdahl;Guttorm Sindre

  • Affiliations:

  • Department of Information Science and Media Studies, University of Bergen, P.O. Box 7800, 5020 Bergen, Norway;Department of Computer and Information Science, Norwegian University of Science and Technology, IDI/NTNU, 7491 Trondheim, Norway

  • Venue:
  • Information and Software Technology
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

A number of methods have been proposed or adapted to include security in the requirements analysis stage, but the industrial take-up has been limited and there are few empirical and comparative evaluations. This paper reports on a pair of controlled experiments that compared two methods for early elicitation of security threats, namely attack trees and misuse cases. The 28 and 35 participants in the two experiments solved two threat identification tasks individually by means of the two techniques, using a Latin-Squares design to control for technique and task order. The dependent variables were effectiveness of the techniques measured as the number of threats found, coverage of the techniques measured in terms of the types of threats found and perceptions of the techniques measured through a post-task questionnaire based on the Technology Acceptance Model. The only difference was that, in the second experiment, the participants were given a pre-drawn use-case diagram to use as a starting point for solving the tasks. In the first experiment, no pre-drawn use-case diagram was provided. The main finding was that attack trees were more effective for finding threats, in particular when there was no pre-drawn use-case diagram. However, the participants had similar opinions of the two techniques, and perception of a technique was not correlated with performance with that technique. The study underlines the need for further comparisons in a broader range of settings involving additional techniques, and it suggests several concrete experiments and other paths for further work.