Factors Influencing Effectiveness in Automated Essay Scoring with LSA

  • Authors:

  • Fridolin Wild;Christina Stahl;Gerald Stermsek;Yoseba Penya;Gustaf Neumann

  • Affiliations:

  • Department of Information Systems and New Media, Vienna University of Economics and Business Administration (WU Wien), Augasse 2-6, A-1090 Vienna, Austria, {firstname.lastname}@wu-wien.ac.at;Department of Information Systems and New Media, Vienna University of Economics and Business Administration (WU Wien), Augasse 2-6, A-1090 Vienna, Austria, {firstname.lastname}@wu-wien.ac.at;Department of Information Systems and New Media, Vienna University of Economics and Business Administration (WU Wien), Augasse 2-6, A-1090 Vienna, Austria, {firstname.lastname}@wu-wien.ac.at;Department of Information Systems and New Media, Vienna University of Economics and Business Administration (WU Wien), Augasse 2-6, A-1090 Vienna, Austria, {firstname.lastname}@wu-wien.ac.at;Department of Information Systems and New Media, Vienna University of Economics and Business Administration (WU Wien), Augasse 2-6, A-1090 Vienna, Austria, {firstname.lastname}@wu-wien.ac.at

  • Venue:
  • Proceedings of the 2005 conference on Artificial Intelligence in Education: Supporting Learning through Intelligent and Socially Informed Technology
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

Automated essay scoring by means of latent semantic analysis (LSA) has recently been subject to increasing interest. Although previous authors have achieved grade ranges similar to those awarded by humans, it is still not clear which and how parameters improve or decrease the effectiveness of LSA. This paper presents an analysis of the effects of these parameters, such as text pre-processing, weighting, singular value dimensionality and type of similarity measure, and benchmarks this effectiveness by comparing machine-assigned with human-assigned scores in a real-world case. We show that each of the identified factors significantly influences the quality of automated essay scoring and that the factors are not independent of each other.