To separate speech: a system for recognizing simultaneous speech

  • Authors:
  • John McDonough;Kenichi Kumatani;Tobias Gehrig;Emilian Stoimenov;Uwe Mayer;Stefan Schacht;Matthias Wölfel;Dietrich Klakow

  • Affiliations:
  • Spoken Language Systems, Saarland University, Saarbrücken, Germany and Institute for Intelligent Sensor-Actuator Systems, University of Karlsruhe, Germany;IDIAP Research Institute, Martigny, Switzerland and Institute for Intelligent Sensor-Actuator Systems, University of Karlsruhe, Germany;Institute for Theoretical Computer Science, University of Karlsruhe, Germany;Institute for Theoretical Computer Science, University of Karlsruhe, Germany;Institute for Theoretical Computer Science, University of Karlsruhe, Germany;Spoken Language Systems, Saarland University, Saarbrücken, Germany;Institute for Theoretical Computer Science, University of Karlsruhe, Germany;Spoken Language Systems, Saarland University, Saarbrücken, Germany

  • Venue:
  • MLMI'07 Proceedings of the 4th international conference on Machine learning for multimodal interaction
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

The PASCAL Speech Separation Challenge (SSC) is based on a corpus of sentences from the Wall Street Journal task read by two speakers simultaneously and captured with two circular eight-channel microphone arrays. This work describes our system for the recognition of such simultaneous speech. Our system has four principal components: A person tracker returns the locations of both active speakers, as well as segmentation information for each utterance, which are often of unequal length; two beamformers in generalized sidelobe canceller (GSC) configuration separate the simultaneous speech by setting their active weight vectors according to a minimum mutual information (MMI) criterion; a postfilter and binary mask operating on the outputs of the beamformers further enhance the separated speech; and finally an automatic speech recognition (ASR) engine based on a weighted finite-state transducer (WFST) returns the most likely word hypotheses for the separated streams. In addition to optimizing each of these components, we investigated the effect of the filter bank design used to perform subband analysis and synthesis during beamforming. On the SSC development data, our system achieved a word error rate of 39.6%.