Benchmarking data mining methods in CAT

  • Authors:
  • Ibrahim Furkan Ince;Adem Karahoca;Dilek Karahoca

  • Affiliations:
  • School of Engineering, The University of Tokyo, Japan;Department of Software Engineering, Bahcesehir University, Istanbul, Turkey;Department of Software Engineering, Bahcesehir University, Istanbul, Turkey

  • Venue:
  • ICIC'11 Proceedings of the 7th international conference on Advanced Intelligent Computing Theories and Applications: with aspects of artificial intelligence
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this study, a ranking test problem of Computer Adaptive Testing (CAT) is benchmarked by employing three popular classifiers: Artificial Neural Network (ANN), Support Vector Machines (SVMs), and Adaptive Network Based Fuzzy Inference System (ANFIS) in terms of ordinal classification performances. As the pilot test, "History of Civilization" class which offered in Bahcesehir University is selected. Item Response Theory (IRT) is focused for the determination of system inputs which are item responses of students, item difficulties of questions, and question levels. Item difficulties of questions are Gaussian normalized to make ordinal decisions. The distance between predicted and expected class values is employed for accuracy estimation. Comparison study is conducted to the ordinal class prediction correctness and performance analysis which is observed by Receiver Operating Characteristic (ROC) graphs. The results show that ANFIS has better performance and higher accuracy than ANN and SVMs in terms of ordinal question classification when the ordinal decisions are practically made by Gaussian Normal Distribution and ROC graphs are focused to observe any significant difference among the performances of classifiers.