PICASSO: automated soundtrack suggestion for multi-modal data

Authors:
Aleksandar Stupar;Sebastian Michel
Affiliations:
Saarland University, Saarbrücken, Germany;Saarland University, Saarbrücken, Germany
Venue:
Proceedings of the 20th ACM international conference on Information and knowledge management
Year:
2011

Citing 9
Cited 0

Locality-sensitive hashing scheme based on p-stable distributions

SCG '04 Proceedings of the twentieth annual symposium on Computational geometry
Comparing Partial Rankings

SIAM Journal on Discrete Mathematics
Emotion-based impressionism slideshow with automatic music accompaniment

Proceedings of the 15th international conference on Multimedia
Information Retrieval for Music and Motion

Information Retrieval for Music and Motion
Music analysis, retrieval and synthesis of audio signals MARSYAS

MM '09 Proceedings of the 17th ACM international conference on Multimedia
Group recommendation: semantics and efficiency

Proceedings of the VLDB Endowment
Toward an automatically generated soundtrack from low-level cross-modal correlations for automotive scenarios

Proceedings of the international conference on Multimedia
Picasso - to sing, you must close your eyes and draw

Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Color and texture descriptors

IEEE Transactions on Circuits and Systems for Video Technology

Quantified Score

Hi-index	0.00

Visualization

Abstract

We demonstrate PICASSO, a novel approach to soundtrack recommendation. Given text, video, or image documents, PICASSO selects the best fitting music pieces, out of a given set of files, for instance, a user's personal mp3 collection. This task, commonly referred to as soundtrack suggestion, is non-trivial as it requires a lot of human attention and a good deal of experience, with master pieces distinguished, e.g., with the Academy Award for Best Original Score. We put forward PICASSO to solve this task in a fully automated way. We address the problem by extracting the required information, in form of music/screenshot samples, from available contemporary movies, making the training set easily obtainable. The training set is further extended with information acquired from movie scripts and subtitles, giving us a richer description of the action and atmosphere expressed in a particular movie scene. Although the number of applications for this approach is very large, we focus on two selected applications. First, we consider recommendation of the soundtrack for the slide show generation based on the given set of images. Second, we consider recommending a soundtrack as the background music for given audio books.