Increasing the utility of quantitative empirical studies for meta-analysis

  • Authors:
  • Heidi Lam;Tamara Munzner

  • Affiliations:
  • University of British Columbia;University of British Columbia

  • Venue:
  • Proceedings of the 2008 Workshop on BEyond time and errors: novel evaLuation methods for Information Visualization
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Despite the long history and consistent use of quantitative empirical methods to evaluate information visualization techniques and systems, our understanding of interface use remains incomplete. While there are inherent limitations to the method, such as the choice of task and data, we believe the utility of study results can be enhanced if they were amenable to meta-analysis. Based on our experience in extracting design guidelines from existing quantitative studies, we recommend improvements to both study design and reporting to promote meta-analysis: (1) Use comparable interfaces in terms of visual elements, information content and amount displayed, levels of data organization displayed, and interaction complexity; (2) Capture usage patterns in addition to overall performance measurements to better identify design tradeoffs; (3) Isolate and study interface factors instead of overall interface performance; and (4) Report more study details, either within the publications, or as supplementary materials.