Automated scoring using a hybrid feature identification technique

  • Authors:
  • Jill Burstein;Karen Kukich;Susanne Wolff;Chi Lu;Martin Chodorow;Lisa Braden-Harder;Mary Dee Harris

  • Affiliations:
  • Educational Testing Service, Princeton, NJ;Educational Testing Service, Princeton, NJ;Educational Testing Service, Princeton, NJ;Educational Testing Service, Princeton, NJ;Hunter College, New York City, NY;Butler-Hill Group, Reston, VA;Language Technology, Inc, Austin, TX

  • Venue:
  • COLING '98 Proceedings of the 17th international conference on Computational linguistics - Volume 1
  • Year:
  • 1998

Quantified Score

Hi-index 0.00

Visualization

Abstract

This study exploits statistical redundancy inherent in natural language to automatically predict scores for essays. We use a hybrid feature identification method, including syntactic structure analysis, rhetorical structure analysis, and topical analysis, to score essay responses from test-takers of the Graduate Management Admissions Test (GMAT) and the Test of Written English (TWE). For each essay question, a stepwise linear regression analysis is run on a training set (sample of human scored essay responses) to extract a weighted set of predictive features for each test question. Score prediction for cross-validation sets is calculated from the set of predictive features. Exact or adjacent agreement between the Electronic Essay Rater (e-rater) score predictions and human rater scores ranged from 87% to 94% across the 15 test questions.