A knowledge representation framework for integration, classification of multi-scale imaging and non-imaging data: preliminary results in predicting prostate cancer recurrence by fusing mass spectrometry and histology

Authors:
George Lee;Scott Doyle;James Monaco;Anant Madabhushi;Michael D. Feldman;Stephen R. Master;John E. Tomaszewski
Affiliations:
Dept of Biomedical Engineering, Rutgers University, Piscataway, New Jersey;Dept of Biomedical Engineering, Rutgers University, Piscataway, New Jersey;Dept of Biomedical Engineering, Rutgers University, Piscataway, New Jersey;Dept of Biomedical Engineering, Rutgers University, Piscataway, New Jersey;Department of Pathology, University of Pennsylvania, Philadelphia, Pennsylvania;Department of Pathology, University of Pennsylvania, Philadelphia, Pennsylvania;Department of Pathology, University of Pennsylvania, Philadelphia, Pennsylvania
Venue:
ISBI'09 Proceedings of the Sixth IEEE international conference on Symposium on Biomedical Imaging: From Nano to Macro
Year:
2009

Citing 3
Cited 1

Sequential Data Fusion via Vector Spaces: Fusion of Heterogeneous Data in the Complex Domain

Journal of VLSI Signal Processing Systems
Investigating the Efficacy of Nonlinear Dimensionality Reduction Schemes in Classifying Gene and Protein Expression Studies

IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB)
Information fusion in biomedical image analysis: combination of data vs. combination of interpretations

IPMI'05 Proceedings of the 19th international conference on Information Processing in Medical Imaging

Computer-aided prognosis: predicting patient and disease outcome via multi-modal image analysis

ISBI'10 Proceedings of the 2010 IEEE international conference on Biomedical imaging: from nano to Macro

Quantified Score

Hi-index	0.00

Visualization

Abstract

The demand for personalized health care requires a wide range of diagnostic tools for determining patient prognosis and theragnosis (response to treatment). These tools present us with data that is both multi-modal (imaging and non-imaging) and multi-scale (proteomics, histology). By utilizing the information in these sources concurrently, we expect significant improvement in predicting patient prognosis and theragnosis. However, a prerequisite to realizing this improvement is the ability to effectively and quantitatively combine information from disparate sources. In this paper, we present a general fusion framework (GFF) aimed towards a combined knowledge representation predicting disease recurrence. To the best of our knowledge, GFF represents the first formal attempt to fuse biomedical image and non-image information directly at the data level as opposed to the decision level, thus preserving more subtle contributions in the original data. GFF represents the different data streams in separate embedding spaces via the application of dimensionality reduction (DR). Data fusion is then implemented by combining the individual reduced embedding spaces. A proof of concept example is considered for evaluating the GFF, whereby protein expression measurements from mass spectrometry are combined with histological image signatures to predict prostate cancer (CaP) recurrence in 6 CaP patients, following therapy. Preliminary results suggest that GFF offers an intelligent way to fuse image and non-image data structures for making prognostic and theragnostic predictions.