An application of methods for the probabilistic three-class classification of pregnancies of unknown location

  • Authors:
  • Ben Van Calster;George Condous;Emma Kirk;Tom Bourne;Dirk Timmerman;Sabine Van Huffel

  • Affiliations:
  • Department of Electrical Engineering (ESAT-SISTA), Katholieke Universiteit Leuven, Kasteelpark Arenberg 10, B-3001 Leuven, Belgium;Early Pregnancy, Acute Gynaecology and Advanced Endosurgery Unit, Nepean Hospital, Nepean Clinical School, University of Sydney, Sydney, Australia;Imperial College London, Hammersmith Campus, Du Cane Road, London, UK;Imperial College London, Hammersmith Campus, Du Cane Road, London, UK and Department of Obstetrics and Gynaecology, University Hospitals Katholieke Universiteit Leuven, Herestraat 49, B-3000 Leuve ...;Department of Obstetrics and Gynaecology, University Hospitals Katholieke Universiteit Leuven, Herestraat 49, B-3000 Leuven, Belgium;Department of Electrical Engineering (ESAT-SISTA), Katholieke Universiteit Leuven, Kasteelpark Arenberg 10, B-3001 Leuven, Belgium

  • Venue:
  • Artificial Intelligence in Medicine
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

Objective: We developed and compared classifiers to predict the outcome of pregnancies of unknown location (PUL). This is a three-class problem, as the possible outcomes are failing PUL, intra-uterine pregnancy (IUP), or ectopic pregnancy (EP). We focused on probabilistic classification because of the importance of uncertainty information in medical decision making. Methods and materials: Nine methods were implemented, based on logistic regression (LR), multi-layer perceptrons, least squares support vector machines (LS-SVMs), and kernel logistic regression (KLR). The LR models involved manual checks for the necessity of input transformations or interaction effects. The classifiers were developed on the training set (n=508) and evaluated on the test set (n=348). We used two performance measures that only evaluate discriminatory potential, and two that investigate the exact probabilities and/or discriminatory potential. Classifier comparison was carried out using a ranking method. Results: The classifier based on a combination of binary LR models using pairwise coupling (LR-PC) ranked first or second for each performance measure. LR-PC obtained test set areas under the receiver operating characteristic curve of 0.989, 0.988, and 0.924 for the prediction of failing PULs, IUPs, and EPs, respectively. Multi-class LR, multi-class KLR, and a combination of binary Bayesian LS-SVMs with radial basis function kernel were always ranked in the top five. Models with low emphasis on nonlinearity were ranked at the bottom. Importantly, LR-PC also performed better than previously developed classifiers based on multi-class LR. Conclusions: The prediction of PULs was good, most notably for failing PULs and IUPs. Logistic regression models performed remarkably well. Multi-class KLR also performed well, whilst nonlinearity was automatically incorporated and probabilistic outputs were directly given without Bayesian analysis or a combination of binary results. The selected inputs are reasonably objective and easy to obtain in clinical practice. Taken together, this study provided useful decision support tools for implementation in clinical practice.