Comparison of oral examination and electronic examination using paired multiple-choice questions

Authors:
Errikos Ventouras;Dimos Triantis;Panagiotis Tsiakas;Charalampos Stergiopoulos
Affiliations:
E-learning Support Team, Technological Educational Institution (T.E.I.) of Athens, 28 Ag. Spyridonos St., Athens, Greece;E-learning Support Team, Technological Educational Institution (T.E.I.) of Athens, 28 Ag. Spyridonos St., Athens, Greece;E-learning Support Team, Technological Educational Institution (T.E.I.) of Athens, 28 Ag. Spyridonos St., Athens, Greece;E-learning Support Team, Technological Educational Institution (T.E.I.) of Athens, 28 Ag. Spyridonos St., Athens, Greece
Venue:
Computers & Education
Year:
2011

Citing 5
Cited 2

The evaluation of electronic marking of examinations

Proceedings of the 8th annual conference on Innovation and technology in computer science education
Evaluating Electronic Examination Methods Applied to Students of Electronics. Effectiveness and Comparison to the Paper-and-Pencil Method.

SUTC '06 Proceedings of the IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing - Vol 2 - Workshops - Volume 02
SIETTE: A Web-Based Tool for Adaptive Testing

International Journal of Artificial Intelligence in Education
Comparison of examination methods based on multiple-choice questions and constructed-response questions using personal computers

Computers & Education
Self-assessment in a feasible, adaptive web-based testing system

IEEE Transactions on Education

Online self-assessment and students' success in higher education institutions

Computers & Education
Collaborative spatial classification

Proceedings of the Fourth International Conference on Learning Analytics And Knowledge

Quantified Score

Hi-index	0.00

Visualization

Abstract

The aim of the present research was to compare the use of multiple-choice questions (MCQs) as an examination method against the oral examination (OE) method. MCQs are widely used and their importance seems likely to grow, due to their inherent suitability for electronic assessment. However, MCQs are influenced by the tendency of examinees to guess answers, warranting research concerning scoring rules different from the simple positive-grades-only scores rule. Alternatively, OE is used in tertiary education, since it enables the assessment of intellectual capabilities and personal traits to a level not found in most other examination formats. However, the significant resource requirements of OE, especially in structured forms, might excessively strain the resources of academic institutions. In the present study, an MCQ test was given to examinees, in the framework of a computer-based learning system. The same examinees took also an OE possessing elements of structure, with three examiners concurrently and independently grading each of the examinees. In the MCQs examination a set of pairs of MCQs was composed. The MCQs in each pair were similar concerning the same topic, but this similarity was not evident for an examinee that did not possess adequate knowledge on the topic addressed in the questions of the pair. The scoring of the paired questions avoided the procedure of mixed-scoring, i.e., both positive and negative markings, while at the same time a pair-wise bonus/penalty scoring rule was adopted. The results of the ''paired'' MCQs examination, when using the pair-wise scoring rule, were statistically indistinguishable with the grades produced by the OE, when made to the same sample of students, on the same topics and with the same levels of difficult. Both the results of the paired MCQs examination, when using the pair-wise scoring rule, and the OE results differed significantly from those obtained by scoring the same MCQs using a positive-grades-only scoring rule that ignored the pairing of MCQs.