Learning Beyond Finite Memory in Recurrent Networks of Spiking Neurons

Authors:
Peter Tiňo;Ashely J. S. Mills
Affiliations:
School of Computer Science, University of Birmingham, Birmingham B15 2TT, U.K.;School of Computer Science, University of Birmingham, Birmingham B15 2TT, U.K.
Venue:
Neural Computation
Year:
2006

Citing 20
Cited 5

Learning and extracting finite state automata with second-order recurrent neural networks

Neural Computation
Learning and extracting initial mealy automata with a modular neural network model

Neural Computation
On the computational power of neural nets

Journal of Computer and System Sciences
Learning the initial state of a second-order recurrent neural network during regular-language inference

Neural Computation
Extraction of rules from discrete-time recurrent neural networks

Neural Networks
Representation of finite state automata in recurrent radial basis function networks

Machine Learning
Spiking neurons

Pulsed neural networks
Pulsed Neural Networks

Pulsed Neural Networks
Introduction To Automata Theory, Languages, And Computation

Introduction To Automata Theory, Languages, And Computation
Synapses as dynamic memory buffers

Neural Networks
Natural Language Grammatical Inference with Recurrent Neural Networks

IEEE Transactions on Knowledge and Data Engineering
Spiking neurons and the induction of finite state machines

Theoretical Computer Science - Natural computing
Real-time computing without stable states: a new framework for neural computation based on perturbations

Neural Computation
Evolution of Spiking Neural Controllers for Autonomous Vision-Based Robots

ER '01 Proceedings of the International Symposium on Evolutionary Robotics From Intelligent Robotics to Artificial Life
From Wheels to Wings with Evolutionary Spiking Circuits

Artificial Life
Rule Extraction from Recurrent Neural Networks: A Taxonomy and Review

Neural Computation
Decoding a Temporal Population Code

Neural Computation
Neural Coding: Higher-Order Temporal Patterns in the Neurostatistics of Cell Assemblies

Neural Computation
The dynamics of discrete-time computation, with application to recurrent neural networks and finite state machine extraction

Neural Computation
Lower bounds for the computational power of networks of spiking neurons

Neural Computation

Fast and adaptive network of spiking neurons for multi-view visual pattern recognition

Neurocomputing
Error-backpropagation in networks of fractionally predictive spiking neurons

ICANN'11 Proceedings of the 21th international conference on Artificial neural networks - Volume Part I
Learning in fully recurrent neural networks by approaching tangent planes to constraint surfaces

Neural Networks
Supervised Learning of Logical Operations in Layered Spiking Neural Networks with Spike Train Encoding

Neural Processing Letters
A supervised multi-spike learning algorithm based on gradient descent for spiking neural networks

Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

We investigate possibilities of inducing temporal structures without fading memory in recurrent networks of spiking neurons strictly operating in the pulse-coding regime. We extend the existing gradient-based algorithm for training feedforward spiking neuron networks, SpikeProp (Bohte, Kok, & La Poutré, 2002), to recurrent network topologies, so that temporal dependencies in the input stream are taken into account. It is shown that temporal structures with unbounded input memory specified by simple Moore machines (MM) can be induced by recurrent spiking neuron networks (RSNN). The networks are able to discover pulse-coded representations of abstract information processing states coding potentially unbounded histories of processed inputs. We show that it is often possible to extract from trained RSNN the target MM by grouping together similar spike trains appearing in the recurrent layer. Even when the target MM was not perfectly induced in a RSNN, the extraction procedure was able to reveal weaknesses of the induced mechanism and the extent to which the target machine had been learned.