Diagnosis of depression by behavioural signals: a multimodal approach

Authors:
Nicholas Cummins;Jyoti Joshi;Abhinav Dhall;Vidhyasaharan Sethu;Roland Goecke;Julien Epps
Affiliations:
The University of New South Wales & National ICT Australia, Sydney, Australia;University of Canberra, Canberra, Australia;Australian National University, Canberra, Australia;The University of New South Wales, Sydney, Australia;University of Canberra / Australian National University, Canberra, Australia;The University of New South Wales & National ICT Australia, Sydney, Australia
Venue:
Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge
Year:
2013

Citing 11
Cited 0

Representing shape with a spatial pyramid kernel

Proceedings of the 6th ACM international conference on Image and video retrieval
Bi-modal emotion recognition from expressive face and body gestures

Journal of Network and Computer Applications
Learning AAM fitting through simulation

Pattern Recognition
GMM-SVM Kernel with a Bhattacharyya-based distance for speaker recognition

IEEE Transactions on Audio, Speech, and Language Processing
Opensmile: the munich versatile and fast open-source audio feature extractor

Proceedings of the international conference on Multimedia
LIBSVM: A library for support vector machines

ACM Transactions on Intelligent Systems and Technology (TIST)
Continuous Prediction of Spontaneous Affect from Multiple Cues and Modalities in Valence-Arousal Space

IEEE Transactions on Affective Computing
Paralinguistics in speech and language-State-of-the-art and the challenge

Computer Speech and Language
Automatic speaker age and gender recognition using acoustic and prosodic level information fusion

Computer Speech and Language
Consistent but modest: a meta-analysis on unimodal and multimodal affect detection accuracies from 30 studies

Proceedings of the 14th ACM international conference on Multimodal interaction
Affective Body Expression Perception and Recognition: A Survey

IEEE Transactions on Affective Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Quantifying behavioural changes in depression using affective computing techniques is the first step in developing an objective diagnostic aid, with clinical utility, for clinical depression. As part of the AVEC 2013 Challenge, we present a multimodal approach for the Depression Sub-Challenge using a GMM-UBM system with three different kernels for the audio subsystem and Space Time Interest Points in a Bag-of-Words approach for the vision subsystem. These are then fused at the feature level to form the combined AV system. Key results include the strong performance of acoustic audio features and the bag-of-words visual features in predicting an individual's level of depression using regression. Interestingly, in the context of the small amount of literature on the subject, is that our feature level multimodal fusion technique is able to outperform both the audio and visual challenge baselines.