Detecting breakdowns in local coherence in the writing of Chinese English learners

  • Authors:

  • Y. Wang;M. Harrington;P. White

  • Affiliations:

  • School of Languages and Comparative Cultural Studies, The University of Queensland, Brisbane, Queensland, Australia and College of Foreign Studies, Jinan University, Guangzhou, China;School of Languages and Comparative Cultural Studies, The University of Queensland, Brisbane, Queensland, Australia;School of Languages and Comparative Cultural Studies, The University of Queensland, Brisbane, Queensland, Australia

  • Venue:
  • Journal of Computer Assisted Learning
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper introduces CTutor, an automated writing evaluation (AWE) tool for detecting breakdowns in local coherence and reports on a study that applies it to the writing of Chinese L2 English learners. The program is based on Centering theory (CT), a theory of local coherence and salience. The principles of CT are first introduced and then the design and function of CTutor are described. The effectiveness and reliability of the program was evaluated in a study that compared performance by CTutor and two human raters on the analysis of local incoherence and provision of revision on learner essays. Intermediate Chinese English as a foreign language learners (n = 52) were divided into two groups: one receiving CTutor feedback and the other receiving feedback from human raters. Learners in both groups completed three essays; each of which involved the submission of a first draft, revision with feedback on local coherence quality and re-submission. Our results from the comparison between CTutor and human experts showed that this software tool is able to detect local coherence breakdowns with moderate accuracy (F1-measure is around 0.4). There was also little difference between participants' responses to CTutor feedback and human feedback in terms of revision behaviour, with both feedback modes resulting in similar revision pattern. Potential use of the program in instructional settings is discussed. © 2012 Wiley Periodicals, Inc.