A comparative evaluation of search techniques for query-by-humming using the MUSART testbed

  • Authors:

  • Roger B. Dannenberg;William P. Birmingham;Bryan Pardo;Ning Hu;Colin Meek;George Tzanetakis

  • Affiliations:

  • Computer Science Department, Carnegie Mellon University, Pittsburgh, PA 15213;Grove City College, Box 3123, Grove City, PA 16127;Department of Electrical Engineering and Computer Science, Northwestern University, Evanston, IL 60208;Google Inc. New York Office, New York, NY 10018;Microsoft Corporation, SQL Server, Building 35/2165, 1 Microsoft Way, Redmond, WA 98052;Computer Science Department, University of Victoria, P.O. Box 3055 STN CSC, Victoria BC V8W 3p6, Canada

  • Venue:
  • Journal of the American Society for Information Science and Technology
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

Query-by-humming systems offer content-based searching for melodies and require no special musical training or knowledge. Many such systems have been built, but there has not been much useful evaluation and comparison in the literature due to the lack of shared databases and queries. The MUSART project testbed allows various search algorithms to be compared using a shared framework that automatically runs experiments and summarizes results. Using this testbed, the authors compared algorithms based on string alignment, melodic contour matching, a hidden Markov model, n-grams, and CubyHum. Retrieval performance is very sensitive to distance functions and the representation of pitch and rhythm, which raises questions about some previously published conclusions. Some algorithms are particularly sensitive to the quality of queries. Our queries, which are taken from human subjects in a realistic setting, are quite difficult, especially for n-gram models. Finally, simulations on query-by-humming performance as a function of database size indicate that retrieval performance falls only slowly as the database size increases. © 2007 Wiley Periodicals, Inc.