"How Do I Evaluate THAT?" Experiences from a Systems-Level Evaluation Effort

  • Authors:

  • Pardha S. Pyla;H. Rex Hartson;Manuel A. Pérez-Quiñones;James D. Arthur;Tonya L. Smith-Jackson;Deborah Hix

  • Affiliations:

  • Center for Human-Computer Interaction, Virginia Tech, Blacksburg, USA 24060;Center for Human-Computer Interaction, Virginia Tech, Blacksburg, USA 24060;Center for Human-Computer Interaction, Virginia Tech, Blacksburg, USA 24060;Center for Human-Computer Interaction, Virginia Tech, Blacksburg, USA 24060;Center for Human-Computer Interaction, Virginia Tech, Blacksburg, USA 24060;Center for Human-Computer Interaction, Virginia Tech, Blacksburg, USA 24060

  • Venue:
  • HCD 09 Proceedings of the 1st International Conference on Human Centered Design: Held as Part of HCI International 2009
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this paper we describe our experience deriving evaluation metrics for a systems-level framework called Ripple that connects software engineering and usability engineering life cycles. This evaluation was conducted with eight teams of graduate students (falling under four types of development models) competing in a joint software engineering and usability engineering course to create a software solution for a real world client. We describe the challenges of evaluating systems-level frameworks and the approach we used to derive metrics given our evaluation context. We conclude with the outcome of this evaluation and the effectiveness of the metrics we employed.