IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use
International Journal of Human-Computer Interaction
The Factor Structure of the System Usability Scale
HCD 09 Proceedings of the 1st International Conference on Human Centered Design: Held as Part of HCI International 2009
User satisfaction from commercial web sites: The effect of design and use
Information and Management
Practical Speech User Interface Design
Practical Speech User Interface Design
CHI '13 Extended Abstracts on Human Factors in Computing Systems
UMUX-LITE: when there's no time for the SUS
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
| Hi-index | 0.01 |
When designing questionnaires there is a tradition of including items with both positive and negative wording to minimize acquiescence and extreme response biases. Two disadvantages of this approach are respondents accidentally agreeing with negative items (mistakes) and researchers forgetting to reverse the scales (miscoding). The original System Usability Scale (SUS) and an all positively worded version were administered in two experiments (n=161 and n=213) across eleven websites. There was no evidence for differences in the response biases between the different versions. A review of 27 SUS datasets found 3 (11%) were miscoded by researchers and 21 out of 158 questionnaires (13%) contained mistakes from users. We found no evidence that the purported advantages of including negative and positive items in usability questionnaires outweigh the disadvantages of mistakes and miscoding. It is recommended that researchers using the standard SUS verify the proper coding of scores and include procedural steps to ensure error-free completion of the SUS by users. Researchers can use the all positive version with confidence because respondents are less likely to make mistakes when responding, researchers are less likely to make errors in coding, and the scores will be similar to the standard SUS.