Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations
Adaptation in natural and artificial systems
Adaptation in natural and artificial systems
Sequential behavior and learning in evolved dynamical neural networks
Adaptive Behavior
SIGGRAPH '94 Proceedings of the 21st annual conference on Computer graphics and interactive techniques
The nature of statistical learning theory
The nature of statistical learning theory
Efficient reinforcement learning through symbiotic evolution
Machine Learning - Special issue on reinforcement learning
Evolving mobile robots in simulated and real environments
Artificial Life
Symbiotic evolution of neural networks in sequential decision tasks
Symbiotic evolution of neural networks in sequential decision tasks
Evolution and Optimum Seeking: The Sixth Generation
Evolution and Optimum Seeking: The Sixth Generation
Numerical Optimization of Computer Models
Numerical Optimization of Computer Models
Designing Neural Networks using Genetic Algorithms
Proceedings of the 3rd International Conference on Genetic Algorithms
Predicting Time Series with Support Vector Machines
ICANN '97 Proceedings of the 7th International Conference on Artificial Neural Networks
Solving Non-Markovian Control Tasks with Neuro-Evolution
IJCAI '99 Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence
Learning precise timing with lstm recurrent networks
The Journal of Machine Learning Research
Robust non-linear control through neuroevolution
Robust non-linear control through neuroevolution
Co-evolving recurrent neurons learn deep memory POMDPs
GECCO '05 Proceedings of the 7th annual conference on Genetic and evolutionary computation
Modeling systems with internal state using evolino
GECCO '05 Proceedings of the 7th annual conference on Genetic and evolutionary computation
Neural Networks - 2005 Special issue: IJCNN 2005
Learning to Forget: Continual Prediction with LSTM
Neural Computation
Neural Computation
Evolino: hybrid neuroevolution / optimal linear search for sequence learning
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Evolving modular fast-weight networks for control
ICANN'05 Proceedings of the 15th international conference on Artificial neural networks: formal models and their applications - Volume Part II
LSTM recurrent networks learn simple context-free and context-sensitive languages
IEEE Transactions on Neural Networks
Gradient calculations for dynamic recurrent neural networks: a survey
IEEE Transactions on Neural Networks
ICANN '08 Proceedings of the 18th international conference on Artificial Neural Networks, Part I
Supervised and Evolutionary Learning of Echo State Networks
Proceedings of the 10th international conference on Parallel Problem Solving from Nature: PPSN X
REINFORCEMENT LEARNING FOR POMDP USING STATE CLASSIFICATION
Applied Artificial Intelligence
Pruning and regularization in reservoir computing
Neurocomputing
Belief propagation in networks of spiking neurons
Neural Computation
Knowledge-based recurrent neural networks in Reinforcement Learning
ASC '07 Proceedings of The Eleventh IASTED International Conference on Artificial Intelligence and Soft Computing
Speeding up reinforcement learning using recurrent neural networks in non-Markovian environments
ASC '07 Proceedings of The Eleventh IASTED International Conference on Artificial Intelligence and Soft Computing
Robust player imitation using multiobjective evolution
CEC'09 Proceedings of the Eleventh conference on Congress on Evolutionary Computation
HyperNEAT controlled robots learn how to drive on roads in simulated environment
CEC'09 Proceedings of the Eleventh conference on Congress on Evolutionary Computation
An EM Based Training Algorithm for Recurrent Neural Networks
ICANN '09 Proceedings of the 19th International Conference on Artificial Neural Networks: Part I
Evolving Memory Cell Structures for Sequence Learning
ICANN '09 Proceedings of the 19th International Conference on Artificial Neural Networks: Part II
Combining Multiple Inputs in HyperNEAT Mobile Agent Controller
ICANN '09 Proceedings of the 19th International Conference on Artificial Neural Networks: Part II
2006: celebrating 75 years of AI - history and outlook: the next 25 years
50 years of artificial intelligence
NEAT in HyperNEAT substituted with genetic programming
ICANNGA'09 Proceedings of the 9th international conference on Adaptive and natural computing algorithms
Recurrent kernel machines: Computing with infinite echo state networks
Neural Computation
Recurrent sparse support vector regression machines trained by active learning in the time-domain
Expert Systems with Applications: An International Journal
Simple deterministically constructed cycle reservoirs with regular jumps
Neural Computation
Survey: Reservoir computing approaches to recurrent neural network training
Computer Science Review
Neural Networks
Modular state space of echo state network
Neurocomputing
Proceedings of the Fourth Symposium on Information and Communication Technology
An automatic method for construction of ensembles to time series prediction
International Journal of Hybrid Intelligent Systems
| Hi-index | 0.00 |
In recent years, gradient-based LSTM recurrent neural networks (RNNs) solved many previously RNN-unlearnable tasks. Sometimes, however, gradient information is of little use for training RNNs, due to numerous local minima. For such cases, we present a novel method: EVOlution of systems with LINear Outputs (Evolino). Evolino evolves weights to the nonlinear, hidden nodes of RNNs while computing optimal linear mappings from hidden state to output, using methods such as pseudo-inverse-based linear regression. If we instead use quadratic programming to maximize the margin, we obtain the first evolutionary recurrent support vector machines. We show that Evolino-based LSTM can solve tasks that Echo State nets (Jaeger, 2004a) cannot and achieves higher accuracy in certain continuous function generation tasks than conventional gradient descent RNNs, including gradient-based LSTM.