A formal framework for linguistic annotation
Speech Communication - Special issue on speech annotation and corpus tools
Journal of Biomedical Informatics
MPLUS: a probabilistic medical language understanding system
BioMed '02 Proceedings of the ACL-02 workshop on Natural language processing in the biomedical domain - Volume 3
Predicting student emotions in computer-human tutoring dialogues
ACL '04 Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics
Annotating attributions and private states
CorpusAnno '05 Proceedings of the Workshop on Frontiers in Corpus Annotations II: Pie in the Sky
Towards role-based filtering of disease outbreak reports
Journal of Biomedical Informatics
Journal of Biomedical Informatics
A methodology for extending domain coverage in SemRep
Journal of Biomedical Informatics
Hi-index | 0.00 |
Evaluating automated indexing applications requires comparing automatically indexed terms against manual reference standard annotations. However, there are no standard guidelines for determining which words from a textual document to include in manual annotations, and the vague task can result in substantial variation among manual indexers. We applied grounded theory to emergency department reports to create an annotation schema representing syntactic and semantic variables that could be annotated when indexing clinical conditions. We describe the annotation schema, which includes variables representing medical concepts (e.g., symptom, demographics), linguistic form (e.g., noun, adjective), and modifier types (e.g., anatomic location, severity). We measured the schema's quality and found: (1) the schema was comprehensive enough to be applied to 20 unseen reports without changes to the schema; (2) agreement between author annotators applying the schema was high, with an F measure of 93%; and (3) the authors made complementary errors when applying the schema, demonstrating that the schema incorporates both linguistic and medical expertise.