SALT: a knowledge acquisition language for propose-and-revise systems
Artificial Intelligence
Towards a technology and a science of machine learning
AI Communications
Fuzzy sets and fuzzy logic: theory and applications
Fuzzy sets and fuzzy logic: theory and applications
Applications of machine learning and rule induction
Communications of the ACM
Knowledge Elicitation: Principles, Techniques and Applications
Knowledge Elicitation: Principles, Techniques and Applications
Machine Learning
Open Mind Common Sense: Knowledge Acquisition from the General Public
On the Move to Meaningful Internet Systems, 2002 - DOA/CoopIS/ODBASE 2002 Confederated International Conferences DOA, CoopIS and ODBASE 2002
Measuring agreement in medical informatics reliability studies
Journal of Biomedical Informatics
Rule Based Expert Systems: The Mycin Experiments of the Stanford Heuristic Programming Project (The Addison-Wesley series in artificial intelligence)
Pattern Recognition and Machine Learning (Information Science and Statistics)
Pattern Recognition and Machine Learning (Information Science and Statistics)
An assessment of group support systems experimental research: methodology and results
Journal of Management Information Systems - Special issue: GSS insights: a look back at the lab, a look forward from the field
Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)
KBS development through ontology mapping and ontology driven acquisition
Proceedings of the 4th international conference on Knowledge capture
The CACHE Study: Group Effects in Computer-supported Collaborative Analysis
Computer Supported Cooperative Work
ACHE: An Architecture for Clinical Hypothesis Examination
CBMS '08 Proceedings of the 2008 21st IEEE International Symposium on Computer-Based Medical Systems
Causality: Models, Reasoning and Inference
Causality: Models, Reasoning and Inference
Unsupervised named-entity extraction from the Web: An experimental study
Artificial Intelligence
Artificial Intelligence in Medicine
International Journal of Human-Computer Studies
| Hi-index | 0.00 |
Objectives: The work reported here focuses on developing novel techniques which enable an expert to detect inconsistencies in 2 (or more) perspectives that the expert might have on the same (classification) task. The high level task which the experts (physicians) had set themselves was to classify, on a 5-point severity scale (A-E), the hourly reports produced by an intensive care unit's patient management system. Method: The INSIGHT system has been developed to support domain experts exploring, and removing inconsistencies in their conceptualization of a task. We report here a study of intensive care physicians reconciling 2 perspectives on their patients. The 2 perspectives provided to INSIGHT were an annotated set of patient records where the expert had selected the appropriate category to describe that snapshot of the patient, and a set of rules which are able to classify the various time points on the same 5-point scale. Inconsistencies between these 2 perspectives are displayed as a confusion matrix; moreover INSIGHT then allows the expert to revise both the annotated datasets (correcting data errors, or changing the assigned categories) and the actual rule-set. Results: Each of the 3 experts achieved a very high degree of consensus (~97%) between his refined knowledge sources (i.e., annotated hourly patient records and the rule-set). We then had the experts produce a common rule-set and then refine their several sets of annotations against it; this again resulted in inter-expert agreements of ~97%. The resulting rule-set can then be used in applications with considerable confidence. Conclusion: This study has shown that under some circumstances, it is possible for domain experts to achieve a high degree of correlation between 2 perspectives of the same task. The experts agreed that the immediate feedback provided by INSIGHT was a significant contribution to this successful outcome.