Regularized online sequential learning algorithm for single-hidden layer feedforward neural networks

Authors:
Hieu Trung Huynh;Yonggwan Won
Affiliations:
Department of Computer Engineering, Chonnam National University, Gwangju 500-757, Republic of Korea and Faculty of Information Technology, HoChiMinh City University of Industry, Viet Nam and NTT I ...;Department of Computer Engineering, Chonnam National University, Gwangju 500-757, Republic of Korea and Korea Bio-IT Foundry Center@Gwangju, Chonnam National University, Gwangju 500-757, Republic ...
Venue:
Pattern Recognition Letters
Year:
2011

Citing 12
Cited 1

A function estimation approach to sequential learning with neural networks

Neural Computation
Matrix computations (3rd ed.)

Matrix computations (3rd ed.)
Model Selection for Regularized Least-Squares Algorithm in Learning Theory

Foundations of Computational Mathematics
Statistical Comparisons of Classifiers over Multiple Data Sets

The Journal of Machine Learning Research
Controlling the parallel layer perceptron complexity using a multiobjective learning algorithm

Neural Computing and Applications
A multi-objective approach to RBF network learning

Neurocomputing
An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network

IEEE Transactions on Information Theory
Methods for choosing the regularization parameter and estimating the noise variance in image restoration and their relation

IEEE Transactions on Image Processing
Performance evaluation of a sequential minimal radial basis function (RBF) neural network learning algorithm

IEEE Transactions on Neural Networks
A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation

IEEE Transactions on Neural Networks
A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks

IEEE Transactions on Neural Networks

Double parallel feedforward neural network based on extreme learning machine with L 1/2 regularizer

Neurocomputing

Quantified Score

Hi-index	0.10

Visualization

Abstract

Online learning algorithms have been preferred in many applications due to their ability to learn by the sequentially arriving data. One of the effective algorithms recently proposed for training single hidden-layer feedforward neural networks (SLFNs) is online sequential extreme learning machine (OS-ELM), which can learn data one-by-one or chunk-by-chunk at fixed or varying sizes. It is based on the ideas of extreme learning machine (ELM), in which the input weights and hidden layer biases are randomly chosen and then the output weights are determined by the pseudo-inverse operation. The learning speed of this algorithm is extremely high. However, it is not good to yield generalization models for noisy data and is difficult to initialize parameters in order to avoid singular and ill-posed problems. In this paper, we propose an improvement of OS-ELM based on the bi-objective optimization approach. It tries to minimize the empirical error and obtain small norm of network weight vector. Singular and ill-posed problems can be overcome by using the Tikhonov regularization. This approach is also able to learn data one-by-one or chunk-by-chunk. Experimental results show the better generalization performance of the proposed approach on benchmark datasets.