Constructive training of recurrent neural networks using hybrid optimization

Authors:
Niranjan Subrahmanya;Yung C. Shin
Affiliations:
School of Mechanical Engineering, Purdue University, West Lafayette, IN 47907, USA;School of Mechanical Engineering, Purdue University, West Lafayette, IN 47907, USA
Venue:
Neurocomputing
Year:
2010

Citing 9
Cited 1

Constructive Backpropagation for Recurrent Networks

Neural Processing Letters
Completely Derandomized Self-Adaptation in Evolution Strategies

Evolutionary Computation
A learning algorithm for continually running fully recurrent neural networks

Neural Computation
A multiobjective genetic algorithm for obtaining the optimal size of a recurrent neural network for grammatical inference

Pattern Recognition
A hybrid of genetic algorithm and particle swarm optimization for recurrent network design

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
An evolutionary algorithm that constructs recurrent neural networks

IEEE Transactions on Neural Networks
Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks

IEEE Transactions on Neural Networks
Constructive learning of recurrent neural networks: limitations of recurrent cascade correlation and a simple solution

IEEE Transactions on Neural Networks
Gradient calculations for dynamic recurrent neural networks: a survey

IEEE Transactions on Neural Networks

A data-based framework for fault detection and diagnostics of non-linear systems with partial state measurement

Engineering Applications of Artificial Intelligence

Quantified Score

Hi-index	0.02

Visualization

Abstract

Training of recurrent neural networks (RNNs) is known to be a very difficult task. This work proposes a novel constructive method for simultaneous structure and parameter training of Elman-type RNNs using a combination of particle swarm optimization (PSO) and covariance matrix adaptation based evolutionary strategy (CMA-ES). The proposed method allows the imposition of certain stability conditions, which can be maintained throughout the constructive process. The examples reported show a monotonic decrease in training error throughout the constructive process and also demonstrate the efficiency of the proposed method for structure and parameter training of RNNs.