Fusion of fMRI, sMRI, and EEG data using canonical correlation analysis

  • Authors:
  • Nicolle M. Correa;Yi-Ou Li;Tulay Adali;Vince D. Calhoun

  • Affiliations:
  • University of Maryland, Baltimore County, Department of CSEE, 21250, USA;University of Maryland, Baltimore County, Department of CSEE, 21250, USA;University of Maryland, Baltimore County, Department of CSEE, 21250, USA;The MIND Research Network and University of New Mexico, Department of ECE, Albuquerque, 87131, USA

  • Venue:
  • ICASSP '09 Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal Processing
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

Typically data acquired through imaging techniques such as functional magnetic resonance imaging (fMRI), structural MRI (sMRI), and electroencephalography (EEG) are analyzed separately. Each modality records brain structure and function at different scales, and fusing information from such complementary modalities promises to provide additional insight into connectivity across brain networks and changes due to disease. Recently, a number of methods have been proposed for data integration and fusion of two brain imaging modalities. We propose a new data fusion scheme based on canonical correlation analysis that enables the detection of associations across multiple modalities. Our multimodal canonical correlation analysis (mCCA) scheme works at the feature level using multi-set CCA to determine inter-subject covariations across modalities. We apply mCCA to fMRI, sMRI, and EEG data collected from patients diagnosed with schizophrenia and healthy controls. Through data collected from an auditory oddball task, we show that the fusion of multiple modalities detects more specific associations as compared to fusion of two modalities.