A computer-aided environment for generating multiple-choice test items

Authors:
Ruslan Mitkov;Le An Ha;Nikiforos Karamanis
Affiliations:
Research Group in Computational Linguistics, School of Humanities, Languages and Social Sciences University of Wolverhampton, Wolverhampton WV1 1SB, UK e-mail: r.mitkov@wlv.ac.uk, l.a.ha@wlv.ac.uk ...;Research Group in Computational Linguistics, School of Humanities, Languages and Social Sciences University of Wolverhampton, Wolverhampton WV1 1SB, UK e-mail: r.mitkov@wlv.ac.uk, l.a.ha@wlv.ac.uk ...;Research Group in Computational Linguistics, School of Humanities, Languages and Social Sciences University of Wolverhampton, Wolverhampton WV1 1SB, UK e-mail: r.mitkov@wlv.ac.uk, l.a.ha@wlv.ac.uk ...
Venue:
Natural Language Engineering
Year:
2006

Citing 12
Cited 18

Assessing agreement on classification tasks: the kappa statistic

Computational Linguistics
Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone

SIGDOC '86 Proceedings of the 5th annual international conference on Systems documentation
A non-projective dependency parser

ANLC '97 Proceedings of the fifth conference on Applied natural language processing
Similarity-based methods for word sense disambiguation

ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
Using syntactic dependency as local context to resolve word sense ambiguity

ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
Computer-aided generation of multiple-choice tests

HLT-NAACL-EDUC '03 Proceedings of the HLT-NAACL 03 workshop on Building educational applications using natural language processing - Volume 2
Automatic question generation for vocabulary assessment

HLT '05 Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing
A web-based system for automatic language skill assessment: EVALING

ASSESSEVALNLP '99 Proceedings of a Symposium on Computer Mediated Language Assessment and Evaluation in Natural Language Processing
Applications of lexical information for algorithmically composing multiple-choice cloze items

EdAppsNLP 05 Proceedings of the second workshop on Building Educational Applications Using NLP
A real-time multiple-choice question generation for language testing: a preliminary study

EdAppsNLP 05 Proceedings of the second workshop on Building Educational Applications Using NLP
Measuring non-native speakers' proficiency of English by using a test with automatically-generated fill-in-the-blank questions

EdAppsNLP 05 Proceedings of the second workshop on Building Educational Applications Using NLP
Using measures of semantic relatedness for word sense disambiguation

CICLing'03 Proceedings of the 4th international conference on Computational linguistics and intelligent text processing

T2D: Generating Dialogues Between Virtual Agents Automatically from Text

IVA '07 Proceedings of the 7th international conference on Intelligent Virtual Agents
Semantic similarity of distractors in multiple-choice tests: extrinsic evaluation

GEMS '09 Proceedings of the Workshop on Geometrical Models of Natural Language Semantics
Generating multiple-choice test items from medical text: a pilot study

INLG '06 Proceedings of the Fourth International Natural Language Generation Conference
Good question! Statistical ranking for question generation

HLT '10 Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics
New issues and solutions in computer-aided design of MCTI and distractors selection for Bulgarian

MRTECEEL '09 Proceedings of the Workshop on Multilingual Resources, Technologies and Evaluation for Central and Eastern European Languages
Language technologies for instructional resources in Bulgarian

ESSLLI'08/09 Proceedings of the 2008 international conference on Interfaces: explorations in logic, language and computation
Bootstrapping multiple-choice tests with THE-MENTOR

CICLing'11 Proceedings of the 12th international conference on Computational linguistics and intelligent text processing - Volume Part I
Unsupervised relation extraction using dependency trees for automatic generation of multiple-choice questions

Canadian AI'11 Proceedings of the 24th Canadian conference on Advances in artificial intelligence
Automatic gap-fill question generation from text books

IUNLPBEA '11 Proceedings of the 6th Workshop on Innovative Use of NLP for Building Educational Applications
Generating example contexts to illustrate a target word sense

IUNLPBEA '11 Proceedings of the 6th Workshop on Innovative Use of NLP for Building Educational Applications
A survey on question answering technology from an information retrieval perspective

Information Sciences: an International Journal
Asking what no one has asked before: using phrase similarities to generate synthetic web search queries

Proceedings of the 20th ACM international conference on Information and knowledge management
A case-based reasoning approach to automating the construction of multiple choice questions

ICCBR'10 Proceedings of the 18th international conference on Case-Based Reasoning Research and Development
Generating educational assessment items from linked open data: the case of DBpedia

ESWC'11 Proceedings of the 8th international conference on The Semantic Web
Mind the gap: learning to choose gaps for question generation

NAACL HLT '12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Generating grammar exercises

Proceedings of the Seventh Workshop on Building Educational Applications Using NLP
Next generation of e-assessment: automatic generation of questions

International Journal of Technology Enhanced Learning
Automatic generation of tourism quiz using blogs

Artificial Life and Robotics

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper describes a novel computer-aided procedure for generating multiple-choice test items from electronic documents. In addition to employing various Natural Language Processing techniques, including shallow parsing, automatic term extraction, sentence transformation and computing of semantic distance, the system makes use of language resources such as corpora and ontologies. It identifies important concepts in the text and generates questions about these concepts as well as multiple-choice distractors, offering the user the option to post-edit the test items by means of a user-friendly interface. In assisting test developers to produce items in a fast and expedient manner without compromising quality, the tool saves both time and production costs.