Evaluating web sites: exploiting user's expectations

• Authors: Maria De Marsico;Stefano Levialdi
• Venue: International Journal of Human-Computer Studies - Incorporating knowledge acquisition
• Year: 2004

A new goal-based approach to measure usability of web sites is presented, strongly taking into account the customer's expectations, which are often hardly foreseeable as a whole. After a general discussion on web site design issues, we present a short survey of evaluation methods currently used for web sites. We next introduce a new taxonomy of site categories in a three-dimensional space, derived from Aristotle's rhetorical triangle, including different aspects of the site designer's goals. In our approach, we use this taxonomy to identify a number of sites belonging to the same category, in order to carry out a comparative analysis of their features. This analysis is the basis for a two-shot generation of a form for the evaluation of that category of sites. In the first shot, the users fill a generic evaluation form, acquainting them with sites characteristics. They are next asked to perform specific tasks of their choice, according to what they expect from a site of the given category. They note their impressions and list those features they found useful; the analysis of their comments is exploited to formulate statements specific to the given category, to be added to the initial form (second shot). We found that the responses to the second, expanded form, provide more comprehensive criteria for site evaluation, and turn helpful to precisely locate flaws in site functionalities. After testing, our methodology has proved very promising and may be applied for the evaluation of any other site category, most of all those providing a set of special services.