Contextual correlates of synonymy
Communications of the ACM
Clustering Algorithms
Determining Semantic Similarity among Entity Classes from Different Ontologies
IEEE Transactions on Knowledge and Data Engineering
Verbs semantics and lexical selection
ACL '94 Proceedings of the 32nd annual meeting on Association for Computational Linguistics
Towards the development of a conceptual distance metric for the UMLS
Journal of Biomedical Informatics
Prospective recruitment of patients with congestive heart failure using an ad-hoc binary classifier
Journal of Biomedical Informatics
Evaluating WordNet-based Measures of Lexical Semantic Relatedness
Computational Linguistics
Inter-patient distance metrics using SNOMED CT defining relationships
Journal of Biomedical Informatics
A semantic approach to IE pattern induction
ACL '05 Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics
Automatic cluster stopping with criterion functions and the gap statistic
NAACL-Demonstrations '06 Proceedings of the 2006 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology: companion volume: demonstrations
Measures of semantic similarity and relatedness in the biomedical domain
Journal of Biomedical Informatics
Empiricism is not a matter of faith
Computational Linguistics
Journal of Biomedical Informatics
Inter-coder agreement for computational linguistics
Computational Linguistics
Robust textual inference via learning and abductive reasoning
AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 3
Using measures of semantic relatedness for word sense disambiguation
CICLing'03 Proceedings of the 4th international conference on Computational linguistics and intelligent text processing
A hybrid knowledge-based and data-driven approach to identifying semantically similar concepts
Journal of Biomedical Informatics
Hi-index | 0.00 |
Our objective is to develop a framework for creating reference standards for functional testing of computerized measures of semantic relatedness. Currently, research on computerized approaches to semantic relatedness between biomedical concepts relies on reference standards created for specific purposes using a variety of methods for their analysis. In most cases, these reference standards are not publicly available and the published information provided in manuscripts that evaluate computerized semantic relatedness measurement approaches is not sufficient to reproduce the results. Our proposed framework is based on the experiences of medical informatics and computational linguistics communities and addresses practical and theoretical issues with creating reference standards for semantic relatedness. We demonstrate the use of the framework on a pilot set of 101 medical term pairs rated for semantic relatedness by 13 medical coding experts. While the reliability of this particular reference standard is in the ''moderate'' range; we show that using clustering and factor analyses offers a data-driven approach to finding systematic differences among raters and identifying groups of potential outliers. We test two ontology-based measures of relatedness and provide both the reference standard containing individual ratings and the R program used to analyze the ratings as open-source. Currently, these resources are intended to be used to reproduce and compare results of studies involving computerized measures of semantic relatedness. Our framework may be extended to the development of reference standards in other research areas in medical informatics including automatic classification, information retrieval from medical records and vocabulary/ontology development.