Evaluation of an architecture for intelligent query and exploration of time-oriented clinical data

  • Authors:

  • Susana B. Martins;Yuval Shahar;Dina Goren-Bar;Maya Galperin;Herbert Kaizer;Lawrence V. Basso;Deborah McNaughton;Mary K. Goldstein

  • Affiliations:

  • Center for Primary Care and Outcomes Research, Stanford University School of Medicine, Stanford, CA, USA and Geriatric Research Educational and Clinical Center (GRECC), VA Palo Alto Health Care Sy ...;Department of Information Systems Engineering, Ben-Gurion University of Negev, Beer-Sheva, Israel;Department of Information Systems Engineering, Ben-Gurion University of Negev, Beer-Sheva, Israel;Department of Information Systems Engineering, Ben-Gurion University of Negev, Beer-Sheva, Israel;Center for Primary Care and Outcomes Research, Stanford University School of Medicine, Stanford, CA, USA;Center for Primary Care and Outcomes Research, Stanford University School of Medicine, Stanford, CA, USA;Center for Primary Care and Outcomes Research, Stanford University School of Medicine, Stanford, CA, USA;Geriatric Research Educational and Clinical Center (GRECC), VA Palo Alto Health Care System, Palo Alto, CA, USA and Center for Primary Care and Outcomes Research, Stanford University School of Med ...

  • Venue:
  • Artificial Intelligence in Medicine
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Objective: Evaluate KNAVE-II, a knowledge-based framework for visualization, interpretation, and exploration of longitudinal clinical data, clinical concepts and patterns. KNAVE-II mediates queries to a distributed temporal-abstraction architecture (IDAN), which uses a knowledge-based problem-solving method specializing in on-the-fly computation of clinical queries. Methods: A two-phase, balanced cross-over study to compare efficiency and satisfaction of a group of clinicians when answering queries of variable complexity about time-oriented clinical data, typical for oncology protocols, using KNAVE-II, versus standard methods: both paper charts and a popular electronic spreadsheet (ESS) in Phase I; an ESS in Phase II. The measurements included the time required to answer and the correctness of answer for each query and each complexity category, and for all queries, assessed versus a predetermined gold standard set by a domain expert. User satisfaction was assessed by the Standard Usability Score (SUS) tool-specific questionnaire and by a ''Usability of Tool Comparison'' comparative questionnaire developed for this study. Results: In both evaluations, subjects answered higher-complexity queries significantly faster using KNAVE-II than when using paper charts or an ESS up to a mean of 255s difference per query versus the ESS for hard queries (p=0.0003) in the second evaluation. Average correctness scores when using KNAVE-II versus paper charts, in the first phase, and the ESS, in the second phase, were significantly higher over all queries. In the second evaluation, 91.6% (110/120) of all of the questions asked within queries of all levels produced correct answers using KNAVE-II, opposed to only 57.5% (69/120) using the ESS (p