The effects of topic familiarity on information search behavior
Proceedings of the 2nd ACM/IEEE-CS joint conference on Digital libraries
Investigating Students' Self-Assessment Skills
UM '01 Proceedings of the 8th International Conference on User Modeling 2001
The effects of domain knowledge on search tactic formulation
Journal of the American Society for Information Science and Technology
Some(what) grand challenges for information retrieval
ACM SIGIR Forum
Self-assessment in Vocabulary Tutoring
ITS '08 Proceedings of the 9th international conference on Intelligent Tutoring Systems
Evaluating the Effect of Open Student Models on Self-Assessment
International Journal of Artificial Intelligence in Education
TREC genomics special issue overview
Information Retrieval
Characterizing the influence of domain expertise on web search behavior
Proceedings of the Second ACM International Conference on Web Search and Data Mining
User-centered evaluation of interactive question answering systems
IQA '06 Proceedings of the Interactive Question Answering Workshop at HLT-NAACL 2006
A data analysis and modelling framework for the evaluation of interactive information retrieval
ECIR'2010 Proceedings of the 32nd European conference on Advances in Information Retrieval
The effects on topic familiarity on online search behaviour and use of relevance criteria
ECIR'06 Proceedings of the 28th European conference on Advances in Information Retrieval
Examining users' knowledge change in the task completion process
Information Processing and Management: an International Journal
Inferring user knowledge level from eye movement patterns
Information Processing and Management: an International Journal
Self-Assessment in the REAP Tutor: Knowledge, Interest, Motivation, & Learning
International Journal of Artificial Intelligence in Education
Hi-index | 0.00 |
Self-assessment of topic/task knowledge is a human metacognitive capacity that impacts information behavior, for example through selection of learning and search strategies. It is often used as a measure in experiments for evaluation of results and those measurements are taken to be generally reliable. We conducted a user study (n=40) to test this by constructing a concept-based topic knowledge representation for each participant and then comparing it with the participant judgment of their topic knowledge elicited with Likert-scale questions. The tasks were in the genomics domain and knowledge representations were constructed from the MeSH thesaurus terms that indexed relevant documents for five topics. The participants rated their familiarity with the topic, the anticipated task difficulty, the amount of learning gained during the task, and made other knowledge-related judgments associated with the task. Although there is considerable variability over individuals, the results provide evidence that these self-assessed topic knowledge measures are correlated in the expected way with the independently-constructed topic knowledge measure. We argue the results provide evidence for the general validity of topic knowledge self-assessment and discuss ways to further explore knowledge self-assessment and its reliability for prediction of individual knowledge levels.