Improving construct validity yields better models of systematic inquiry, even with less information

Authors:
Michael A. Sao Pedro;Ryan S. J. d. Baker;Janice D. Gobert
Affiliations:
Learning Sciences and Technologies Program, Worcester Polytechnic Institute;Learning Sciences and Technologies Program, Worcester Polytechnic Institute;Learning Sciences and Technologies Program, Worcester Polytechnic Institute
Venue:
UMAP'12 Proceedings of the 20th international conference on User Modeling, Adaptation, and Personalization
Year:
2012

Citing 10
Cited 1

C4.5: programs for machine learning

C4.5: programs for machine learning
Floating search methods in feature selection

Pattern Recognition Letters
Hybrid Genetic Algorithms for Feature Selection

IEEE Transactions on Pattern Analysis and Machine Intelligence
Case studies in the use of ROC curve analysis for sensor-based estimates in human computer interaction

GI '05 Proceedings of Graphics Interface 2005
Using log files to track students' model-based inquiry

ICLS '06 Proceedings of the 7th international conference on Learning sciences
Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)

Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)
About the relationship between ROC curves and Cohen's kappa

Engineering Applications of Artificial Intelligence
Helping students make controlled experiments more informative

ICLS '10 Proceedings of the 9th International Conference of the Learning Sciences - Volume 1
Discovering and recognizing student interaction patterns in exploratory learning environments

ITS'10 Proceedings of the 10th international conference on Intelligent Tutoring Systems - Volume Part I
Leveraging machine-learned detectors of systematic inquiry behavior to estimate and predict transfer of inquiry skill

User Modeling and User-Adapted Interaction

What different kinds of stratification can reveal about the generalizability of data-mined skill assessment models

Proceedings of the Third International Conference on Learning Analytics and Knowledge

Quantified Score

Hi-index	0.00

Visualization

Abstract

Data-mined models often achieve good predictive power, but sometimes at the cost of interpretability. We investigate here if selecting features to increase a model's construct validity and interpretability also can improve the model's ability to predict the desired constructs. We do this by taking existing models and reducing the feature set to increase construct validity. We then compare the existing and new models on their predictive capabilities within a held-out test set in two ways. First, we analyze the models' overall predictive performance. Second, we determine how much student interaction data is necessary to make accurate predictions. We find that these reduced models with higher construct validity not only achieve better agreement overall, but also achieve better prediction with less data. This work is conducted in the context of developing models to assess students' inquiry skill at designing controlled experiments and testing stated hypotheses within a science inquiry microworld.