γ-C plane and robustness in static reservoir for nonlinear regression estimation

Authors:
Zhiwei Shi;Min Han
Affiliations:
School of Electronic and Information Engineering, Dalian University of Technology, Liaoning 116023, China;School of Electronic and Information Engineering, Dalian University of Technology, Liaoning 116023, China
Venue:
Neurocomputing
Year:
2009

Citing 25
Cited 1

The truncated SVD as a method for regularization

BIT
A training algorithm for optimal margin classifiers

COLT '92 Proceedings of the fifth annual workshop on Computational learning theory
Training with noise is equivalent to Tikhonov regularization

Neural Computation
Efficient SVM Regression Training with SMO

Machine Learning
Real-time computing without stable states: a new framework for neural computation based on perturbations

Neural Computation
A tutorial on support vector regression

Statistics and Computing
Decomposition methods for linear support vector machines

Neural Computation
Efficient Model Selection for Kernel Logistic Regression

ICPR '04 Proceedings of the Pattern Recognition, 17th International Conference on (ICPR'04) Volume 2 - Volume 02
Fast exact leave-one-out cross-validation of sparse least-squares support vector machines

Neural Networks
Pattern Recognition and Machine Learning (Information Science and Statistics)

Pattern Recognition and Machine Learning (Information Science and Statistics)
Parallel Programming in C with MPI and OpenMP

Parallel Programming in C with MPI and OpenMP
Training Recurrent Networks by Evolino

Neural Computation
Editorial: Special issue on echo state networks and liquid state machines

Neural Networks
2007 Special Issue: Online reservoir adaptation by intrinsic plasticity for backpropagation-decorrelation and echo state learning

Neural Networks
Multicategory Classification Using An Extreme Learning Machine for Microarray Gene Expression Cancer Diagnosis

IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB)
Letters: Convex incremental extreme learning machine

Neurocomputing
Recursive DLS solution for extreme learning machine-based channel equalizer

Neurocomputing
Enhanced random search based incremental extreme learning machine

Neurocomputing
A protein secondary structure prediction framework based on the Extreme Learning Machine

Neurocomputing
Noise-Robust Automatic Speech Recognition Using a Predictive Echo State Network

IEEE Transactions on Audio, Speech, and Language Processing
The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network

IEEE Transactions on Information Theory
An introduction to kernel-based learning algorithms

IEEE Transactions on Neural Networks
Robust support vector regression networks for function approximation with outliers

IEEE Transactions on Neural Networks
Universal approximation using incremental constructive feedforward networks with random hidden nodes

IEEE Transactions on Neural Networks
Support Vector Echo-State Machine for Chaotic Time-Series Prediction

IEEE Transactions on Neural Networks

Control of discrete chaotic systems based on echo state network modeling with an adaptive noise canceler

Knowledge-Based Systems

Quantified Score

Hi-index	0.01

Visualization

Abstract

Reservoir method is applied to the feed-forward learning machines for nonlinear regression estimation. Inspired by the existing experience from extreme learning machine (ELM), the new method inherits the basic idea from support vector echo-state machines, but eliminates the internal feedback matrix to adapt for the feed-forward usage. Based on the analysis of nonlinearity in reservoir and regularization in readout weights, the parameters of input scaling and penalty regularization are taken as the hyper-parameters to characterize a static reservoir (ELM), and then a proper reservoir is identified on the @c-C plane based on a generalization error criterion. For outlier suppression, the regularized robust regression is applied in the reservoir feature space, and it leads to an efficient algorithm for large-scale problems, which can be solved by Cholesky decomposition. The proposed method is compared with the classical kernel method and ELM method on several benchmark nonlinear regression datasets, and the results indicate the method is comparable with the existing methods.