A new quantitative quality measure for machine translation systems
COLING '92 Proceedings of the 14th conference on Computational linguistics - Volume 2
BLEU: a method for automatic evaluation of machine translation
ACL '02 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
EdAppsNLP 05 Proceedings of the second workshop on Building Educational Applications Using NLP
| Hi-index | 0.00 |
Assisting in foreign language learning is one of the major areas in which natural language processing technology can contribute. This paper proposes a computerized method of measuring communicative skill in English as a foreign language. The proposed method consists of two parts. The first part involves a test sentence selection part to achieve precise measurement with a small test set. The second part is the actual measurement, which has three steps. Step one asks proficiency-known human subjects to translate Japanese sentences into English. Step two gauges the match between the translations of the subjects and correct translations based on the n-gram overlap or the edit distance between translations. Step three learns the relationship between proficiency and match. By regression it finds a straight-line fitting for the scatter plot representing the proficiency and matches of the subjects. Then, it estimates proficiency of proficiency-unknown users by using the line and the match. Based on this approach, we conducted experiments on estimating the Test of English for International Communication (TOEIC) score. We collected two sets of data consisting of English sentences translated from Japanese. The first set consists of 330 sentences, each translated to English by 29 subjects with varied English proficiency. The second set consists of 510 sentences translated in a similar manner by a separate group of 18 subjects. We found that the estimated scores correlated with the actual scores.