A fast learning algorithm for time-delay neural networks

Authors:
Minghu Jiang;Georges Gielen;Beixing Deng;Xiaoyan Zhu
Affiliations:
Department of Chinese Language, Lab. of Computational Linguistics, Tsinghua University, Beijing 100084, People's Republic of China and Departement Elektrotechniek, ESAT-MICAS, K.U. Leuven, Kasteel ...;Departement Elektrotechniek, ESAT-MICAS, K.U. Leuven, Kasteelpark Arenberg 10, 3001 Leuven, Belgium;Department of Electronics Engineering, Tsinghua University, Beijing 100084, People's Republic of China;State Key Lab of Intelligent Tech. & Systems, Department of Computer, Tsinghua University, Beijing 100084, People's Republic of China
Venue:
Information Sciences—Applications: An International Journal
Year:
2002

Citing 3
Cited 4

A speaker identification system using a model of artificial neural networks for an elevator application

Information Sciences—Informatics and Computer Science: An International Journal
Accelerating the training of feedforward neural networks using generalized Hebbian rules for initializing the internal representations

IEEE Transactions on Neural Networks
Efficient classification for multiclass problems using modular neural networks

IEEE Transactions on Neural Networks

Global convergence of neural networks with mixed time-varying delays and discontinuous neuron activations

Information Sciences: an International Journal
On the periodic dynamics of a class of time-varying delayed neural networks via differential inclusions

Neural Networks
On the periodic dynamics of a general Cohen-Grossberg BAM neural networks via differential inclusions

Neurocomputing
Periodicity and global exponential stability of generalized Cohen-Grossberg neural networks with discontinuous activations and mixed delays

Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

To solve the long training time in Waibel's time-delay neural networks (TDNNs) for phoneme recognition, several improved fast learning methods are put forward: (1) by a combination between the unsupervised Oja's rule learning method with the similar error backpropagation (BP) algorithm for initial weights training; (2) by improving of the error energy function with weights update according to the output error size; (3) by changing of BP error from along layers to frames and using the averaged overlapping part of frame-shift delta values in the weights of the bottom layer; (4) by training of the data from a small to large number of samples gradually; and (5) by using the optimal modular neural networks (OMNNs) with tree structure for multiple phonemes. Our experimental results indicate that the convergence speed is accelerated with orders of magnitude and in most cases the error function descends monotonically while the network complexity increases less and the recognition rates are almost the same among the different comparative experiments.