Vocabulary profile as a measure of vocabulary sophistication

Authors:
Su-Youn Yoon;Suma Bhat;Klaus Zechner
Affiliations:
Educational Testing Service, Princeton, NJ;University of Illinois, Urbana-Champaign, IL;Educational Testing Service, Princeton, NJ
Venue:
Proceedings of the Seventh Workshop on Building Educational Applications Using NLP
Year:
2012

Citing 6
Cited 0

Automatic scoring of pronunciation quality

Speech Communication
Automatic Pronunciation Scoring for Language Instruction

ICASSP '97 Proceedings of the 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '97)-Volume 2 - Volume 2
Automatic scoring of non-native spontaneous speech in tests of spoken English

Speech Communication
Computing and evaluating syntactic complexity features for automated scoring of spontaneous non-native speech

HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1
Detecting structural events for assessing non-native speech

IUNLPBEA '11 Proceedings of the 6th Workshop on Innovative Use of NLP for Building Educational Applications
Exploring content features for automated speech scoring

NAACL HLT '12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

Quantified Score

Hi-index	0.00

Visualization

Abstract

This study presents a method that assesses ESL learners' vocabulary usage to improve an automated scoring system of spontaneous speech responses by non-native English speakers. Focusing on vocabulary sophistication, we estimate the difficulty of each word in the vocabulary based on its frequency in a reference corpus and assess the mean difficulty level of the vocabulary usage across the responses (vocabulary profile). Three different classes of features were generated based on the words in a spoken response: coverage-related, average word rank and the average word frequency and the extent to which they influence human-assigned language proficiency scores was studied. Among these three types of features, the average word frequency showed the most predictive power. We then explored the impact of vocabulary profile features in an automated speech scoring context, with particular focus on the impact of two factors: genre of reference corpora and the characteristics of item-types. The contribution of the current study lies in the use of vocabulary profile as a measure of lexical sophistication for spoken language assessment, an aspect heretofore unexplored in the context of automated speech scoring.