Improving articulatory feature and phoneme recognition using multitask learning

  • Authors:
  • Ramya Rasipuram;Mathew Magimai-Doss

  • Affiliations:
  • Idiap Research Institute, Martigny, Switzerland and Ecole Polytechnique Fédérale de Lausanne, Switzerland;Idiap Research Institute, Martigny, Switzerland

  • Venue:
  • ICANN'11 Proceedings of the 21th international conference on Artificial neural networks - Volume Part I
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Speech sounds can be characterized by articulatory features. Articulatory features are typically estimated using a set of multilayer perceptrons (MLPs), i.e., a separate MLP is trained for each articulatory feature. In this paper, we investigate multitask learning (MTL) approach for joint estimation of articulatory features with and without phoneme classification as subtask. Our studies show that MTL MLP can estimate articulatory features compactly and efficiently by learning the inter-feature dependencies through a common hidden layer representation. Furthermore, adding phoneme as subtask while estimating articulatory features improves both articulatory feature estimation and phoneme recognition. On TIMIT phoneme recognition task, articulatory feature posterior probabilities obtained by MTL MLP achieve a phoneme recognition accuracy of 73.2%, while the phoneme posterior probabilities achieve an accuracy of 74.0%.