The weighted majority algorithm
Information and Computation
A maximum entropy approach to natural language processing
Computational Linguistics
IEEE Transactions on Pattern Analysis and Machine Intelligence
Experimental evaluation of expert fusion strategies
Pattern Recognition Letters - Special issue on pattern recognition in practice VI
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Natural Language Engineering
Term identification in the biomedical literature
Journal of Biomedical Informatics - Special issue: Named entity recognition in biomedicine
MPLUS: a probabilistic medical language understanding system
BioMed '02 Proceedings of the ACL-02 workshop on Natural language processing in the biomedical domain - Volume 3
On naive Bayesian fusion of dependent classifiers
Pattern Recognition Letters
Incorporating non-local information into information extraction systems by Gibbs sampling
ACL '05 Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics
Adaptive fusion and co-operative training for classifier ensembles
Pattern Recognition
Overview of BioNLP'09 shared task on event extraction
BioNLP '09 Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing: Shared Task
Methodological Review: What can natural language processing do for clinical decision support?
Journal of Biomedical Informatics
MCS'03 Proceedings of the 4th international conference on Multiple classifier systems
Comparing and combining chunkers of biomedical text
Journal of Biomedical Informatics
Clinical entity recognition using structural support vector machines with rich features
Proceedings of the ACM sixth international workshop on Data and text mining in biomedical informatics
Unsupervised biomedical named entity recognition: Experiments with clinical and biological texts
Journal of Biomedical Informatics
Hi-index | 0.00 |
Recognition of medical concepts is a basic step in information extraction from clinical records. We wished to improve on the performance of a variety of concept recognition systems by combining their individual results. We selected two dictionary-based systems and five statistical-based systems that were trained to annotate medical problems, tests, and treatments in clinical records. Manually annotated clinical records for training and testing were made available through the 2010 i2b2/VA (Informatics for Integrating Biology and the Bedside) challenge. Results of individual systems were combined by a simple voting scheme. The statistical systems were trained on a set of 349 records. Performance (precision, recall, F-score) was assessed on a test set of 477 records, using varying voting thresholds. The combined annotation system achieved a best F-score of 82.2% (recall 81.2%, precision 83.3%) on the test set, a score that ranks third among 22 participants in the i2b2/VA concept annotation task. The ensemble system had better precision and recall than any of the individual systems, yielding an F-score that is 4.6% point higher than the best single system. Changing the voting threshold offered a simple way to obtain a system with high precision (and moderate recall) or one with high recall (and moderate precision). The ensemble-based approach is straightforward and allows the balancing of precision versus recall of the combined system. The ensemble system is freely available and can easily be extended, integrated in other systems, and retrained.