Practical Recurrent Learning (PRL) in the Discrete Time Domain

Authors:
Mohamad Faizal Samsudin;Takeshi Hirose;Katsunari Shibata
Affiliations:
Department of Electrical and Electronic Engineering, Oita University, Oita, Japan 870-1192;Department of Electrical and Electronic Engineering, Oita University, Oita, Japan 870-1192;Department of Electrical and Electronic Engineering, Oita University, Oita, Japan 870-1192
Venue:
Neural Information Processing
Year:
2007

Citing 3
Cited 0

Learning internal representations by error propagation

Parallel distributed processing: explorations in the microstructure of cognition, vol. 1
Long short-term memory

Neural Computation
A learning algorithm for continually running fully recurrent neural networks

Neural Computation

Quantified Score

Hi-index	0.00

Visualization

Abstract

One of the authors has proposed a simple learning algorithm for recurrent neural networks, which requires computational cost and memory capacity in practical order O(n2)[1]. The algorithm was formulated in the continuous time domain, and it was shown that a sequential NAND problem was successfully learned by the algorithm. In this paper, the authors name the learning "Practical Recurrent Learning (PRL)", and the learning algorithm is simplified and converted in the discrete time domain for easy analysis. It is shown that sequential EXOR problem and 3-bit parity problem as non linearly-separable problems can be learned by PRL even though the learning performance is often quite inferior to BPTT that is one of the most popular learning algorithms for recurrent neural networks. Furthermore, the learning process is observed and the character of PRL is shown.