A comparison of usage evaluation and inspection methods for assessing groupware usability

  • Authors:

  • Michelle Potts Steves;Emile Morse;Carl Gutwin;Saul Greenberg

  • Affiliations:

  • National Institute of Standards and Technology, Gaithersburg, Maryland;National Institute of Standards and Technology, Gaithersburg, Maryland;University of Saskatchewan, Canada;University of Calgary, Canada

  • Venue:
  • GROUP '01 Proceedings of the 2001 International ACM SIGGROUP Conference on Supporting Group Work
  • Year:
  • 2001

Quantified Score

Hi-index 0.00

Visualization

Abstract

Many researchers believe that groupware can only be evaluated by studying real collaborators in their real contexts, a process that tends to be expensive and time-consuming. Others believe that it is more practical to evaluate groupware through usability inspection methods. Deciding between these two approaches is difficult, because it is unclear how they compare in a real evaluation situation. To address this problem, we carried out a dual evaluation of a groupware system, with one evaluation applying user-based techniques, and the other using inspection methods. We compared the results from the two evaluations and concluded that, while the two methods have their own strengths, weaknesses, and trade-offs, they are complementary. Because the two methods found overlapping problems, we expect that they can be used in tandem to good effect, e.g., applying the discount method prior to a field study, with the expectation that the system deployed in the more expensive field study has a better chance of doing well because some pertinent usability problems will have already been addressed.