Analyzing Ordinal Scales in Studies of Virtual Environments: Likert or Lump It!

Authors:
Henry J. Gardner;Michael A. Martin
Affiliations:
Department of Computer Science, FEIT College of Engineering and Computer Science, Australian National University, Canberra, ACT 0200 Australia;School of Finance and Applied Statistics, College of Business and Economics, Australian National University, Canberra, ACT 0200 Australia
Venue:
Presence: Teleoperators and Virtual Environments
Year:
2007

Citing 1
Cited 3

The responses of people to virtual humans in an immersive virtual environment

Presence: Teleoperators and Virtual Environments - Special issue: Collaborative information visualization environments

The Use of Questionnaire Data in Presence Studies: Do Not Seriously Likert

Presence: Teleoperators and Virtual Environments
Simulating virtual environments within virtual environments as the basis for a psychophysics of presence

ACM SIGGRAPH 2010 papers
VirSchool: The effect of background music and immersive display systems on memory for facts learned in an educational virtual environment

Computers & Education

Quantified Score

Hi-index	0.00

Visualization

Abstract

Likert scaled data, which are frequently collected in studies of interaction in virtual environments, demand specialized statistical tools for analysis. The routine use of statistical methods appropriate for continuous data in this context can lead to significant inferential flaws. Likert scaled data are ordinal rather than interval scaled and need to be analyzed using rank based statistical procedures that are widely available. Likert scores are “lumpy” in the sense that they cluster around a small number of fixed values. This lumpiness is made worse by the tendency for subjects to cluster towards either the middle or the extremes of the scale. We suggest an ad hoc method to deal with such data which can involve a further lumping of the results followed by the application of nonparametric statistics. Averaging Likert scores over several different survey questions, which is sometimes done in studies of interaction in virtual environments, results in a different sort of lumpiness. The lumped variables which are obtained in this manner can be quite murky and should be used with great caution, if at all, particularly if the number of questions over which such averaging is carried out is small.