Universal approximation using incremental constructive feedforward networks with random hidden nodes

Authors:
Guang-Bin Huang;Lei Chen;Chee-Kheong Siew
Affiliations:
Sch. of Electr. & Electron. Eng., Nanyang Technol. Univ., Singapore, Singapore;-;-
Venue:
IEEE Transactions on Neural Networks
Year:
2006

Citing 0
Cited 140

Letters: Adaptive neural network control of uncertain nonlinear systems with nonsmooth actuator nonlinearities

Neurocomputing
Multicategory Classification Using An Extreme Learning Machine for Microarray Gene Expression Cancer Diagnosis

IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB)
Letters: Convex incremental extreme learning machine

Neurocomputing
Linear programming minimum sphere set covering for extreme learning machines

Neurocomputing
Incremental extreme learning machine with fully complex hidden nodes

Neurocomputing
Recursive DLS solution for extreme learning machine-based channel equalizer

Neurocomputing
Error-driven active learning in growing radial basis function networks for early robot learning

Neurocomputing
Letters: Neural networks stabilization and disturbance attenuation for nonlinear switched impulsive systems

Neurocomputing
Deterministic neural classification

Neural Computation
Maximizing area under ROC curve for biometric scores fusion

Pattern Recognition
Enhanced random search based incremental extreme learning machine

Neurocomputing
Universal Approximation and QoS Violation Application of Extreme Learning Machine

Neural Processing Letters
Ensembling Extreme Learning Machines

ISNN '07 Proceedings of the 4th international symposium on Neural Networks: Advances in Neural Networks
A Robust Online Sequential Extreme Learning Machine

ISNN '07 Proceedings of the 4th international symposium on Neural Networks: Advances in Neural Networks
An Improved On-Line Sequential Learning Algorithm for Extreme Learning Machine

ISNN '07 Proceedings of the 4th international symposium on Neural Networks: Advances in Neural Networks
Integrated Analytic Framework for Neural Network Construction

ISNN '07 Proceedings of the 4th international symposium on Neural Networks: Part II--Advances in Neural Networks
Design and Implementation of a General Purpose Neural Network Processor

ISNN '07 Proceedings of the 4th international symposium on Neural Networks: Part II--Advances in Neural Networks
Active Noise Control Using a Feedforward Network with Online Sequential Extreme Learning Machine

ISNN '08 Proceedings of the 5th international symposium on Neural Networks: Advances in Neural Networks
A machine learning methodology for the analysis of workplace accidents

International Journal of Computer Mathematics - Recent Advances in Computational and Applied Mathematics in Science and Engineering
Convergence analysis of convex incremental neural networks

Annals of Mathematics and Artificial Intelligence
Sales forecasting using extreme learning machine with applications in fashion retailing

Decision Support Systems
A fast pruned-extreme learning machine for classification problem

Neurocomputing
Incremental constructive ridgelet neural network

Neurocomputing
Evolutionary product-unit neural networks classifiers

Neurocomputing
A protein secondary structure prediction framework based on the Extreme Learning Machine

Neurocomputing
Simplified neural networks algorithm for function approximation on discrete input spaces in high dimension-limited sample applications

Neurocomputing
Neural Network with Matrix Inputs

Informatica
Brief paper: An adaptive optimization scheme with satisfactory transient performance

Automatica (Journal of IFAC)
γ-C plane and robustness in static reservoir for nonlinear regression estimation

Neurocomputing
Systemical convergence rate analysis of convex incremental feedforward neural networks

Neurocomputing
Partial Lanczos extreme learning machine for single-output regression problems

Neurocomputing
Letters: Ensemble of online sequential extreme learning machine

Neurocomputing
Online sequential fuzzy extreme learning machine for function approximation and classification problems

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics - Special issue on cybernetics and cognitive informatics
Two-stage mixed discrete-continuous identification of radial basis function (RBF) neural models for nonlinear systems

IEEE Transactions on Circuits and Systems Part I: Regular Papers
The graph neural network model

IEEE Transactions on Neural Networks
Lattice dynamical wavelet neural networks implemented using particle swarm optimization for spatio-temporal system identification

IEEE Transactions on Neural Networks
Large scale nonlinear control system fine-tuning through learning

IEEE Transactions on Neural Networks
Error minimized extreme learning machine with growth of hidden nodes and incremental learning

IEEE Transactions on Neural Networks
Almost Random Projection Machine

ICANN '09 Proceedings of the 19th International Conference on Artificial Neural Networks: Part I
Efficient Parametric Adjustment of Fuzzy Inference System Using Error Backpropagation Method

ICANN '09 Proceedings of the 19th International Conference on Artificial Neural Networks: Part I
A new neural network-based approach for self-tuning control of nonlinear multi-input multi-output dynamic systems

ACC'09 Proceedings of the 2009 conference on American Control Conference
A constructive enhancement for online sequential extreme learning machine

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
Asymptotic tracking control scheme for mechanical systems with external disturbances and friction

Neurocomputing
Composite function wavelet neural networks with extreme learning machine

Neurocomputing
Intelligent approaches using support vector machine and extreme learning machine for transmission line protection

Neurocomputing
A division algebraic framework for multidimensional support vector regression

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
OP-ELM: optimally pruned extreme learning machine

IEEE Transactions on Neural Networks
A new online learning algorithm for structure-adjustable extreme learning machine

Computers & Mathematics with Applications
Performance enhancement of extreme learning machine for multi-category sparse data classification problems

Engineering Applications of Artificial Intelligence
Constructive approximation to multivariate function by decay RBF neural network

IEEE Transactions on Neural Networks
Two-stage extreme learning machine for regression

Neurocomputing
Constructive hidden nodes selection of extreme learning machine for regression

Neurocomputing
Incremental-based extreme learning machine algorithms for time-variant neural networks

ICIC'10 Proceedings of the 6th international conference on Advanced intelligent computing theories and applications: intelligent computing
An adaptive wavelet neural network for spatio-temporal system identification

Neural Networks
Optimization method based extreme learning machine for classification

Neurocomputing
Ordinal extreme learning machine

Neurocomputing
Approximation capability of interpolation neural networks

Neurocomputing
Approximation of functions with spatial inhomogeneity based on "true" ortho-ridgelet neural network

Applied Soft Computing
The multidimensional function approximation based on constructive wavelet RBF neural network

Applied Soft Computing
Application of extreme learning machine for series compensated transmission line protection

Engineering Applications of Artificial Intelligence
Extreme and incremental learning based single-hidden-layer regularization ridgelet network

Neurocomputing
ELM-Based time-variant neural networks with incremental number of output basis functions

ISNN'11 Proceedings of the 8th international conference on Advances in neural networks - Volume Part I
Extreme learning machine with adaptive growth of hidden nodes and incremental updating of output weights

AIS'11 Proceedings of the Second international conference on Autonomous and intelligent systems
Face recognition based on kernelized extreme learning machine

AIS'11 Proceedings of the Second international conference on Autonomous and intelligent systems
Batch intrinsic plasticity for extreme learning machines

ICANN'11 Proceedings of the 21th international conference on Artificial neural networks - Volume Part I
Weighting Efficient Accuracy and Minimum Sensitivity for Evolving Multi-Class Classifiers

Neural Processing Letters
Parameter estimation of q-Gaussian Radial Basis Functions Neural Networks with a Hybrid Algorithm for binary classification

Neurocomputing
Voting based extreme learning machine

Information Sciences: an International Journal
Comparing error minimized extreme learning machines and support vector sequential feed-forward neural networks

Neural Networks
Channel equalization using complex extreme learning machine with RBF kernels

ISNN'06 Proceedings of the Third international conference on Advances in Neural Networks - Volume Part III
Letters: Daily maximum load forecasting of consecutive national holidays using OSELM-based multi-agents system with weighted average strategy

Neurocomputing
Reversible watermarking via extreme learning machine prediction

Neurocomputing
Enhanced extreme learning machine with modified gram-schmidt algorithm

ISNN'10 Proceedings of the 7th international conference on Advances in Neural Networks - Volume Part I
Orthogonal least squares based on singular value decomposition for spare basis selection

ISNN'10 Proceedings of the 7th international conference on Advances in Neural Networks - Volume Part I
Evolutionary learning using a sensitivity-accuracy approach for classification

HAIS'10 Proceedings of the 5th international conference on Hybrid Artificial Intelligence Systems - Volume Part II
2012 Special Issue: Identification of brain regions responsible for Alzheimer's disease using a Self-adaptive Resource Allocation Network

Neural Networks
Novel weighting in single hidden layer feedforward neural networks for data classification

Computers & Mathematics with Applications
Letters: Random optimized geometric ensembles

Neurocomputing
A comparative analysis of support vector machines and extreme learning machines

Neural Networks
A new automatic target recognition system based on wavelet extreme learning machine

Expert Systems with Applications: An International Journal
Modeling spectral data based on mutual information and kernel extreme learning machines

ISNN'12 Proceedings of the 9th international conference on Advances in Neural Networks - Volume Part I
A novel approach for detection and classification of mammographic microcalcifications using wavelet analysis and extreme learning machine

Computers in Biology and Medicine
Applying least angle regression to ELM

Canadian AI'12 Proceedings of the 25th Canadian conference on Advances in Artificial Intelligence
A novel locally regularized automatic construction method for RBF neural models

Neurocomputing
A Computer Aided Diagnosis System for Thyroid Disease Using Extreme Learning Machine

Journal of Medical Systems
A structure optimisation algorithm for feedforward neural network construction

Neurocomputing
Fashion retailing forecasting based on extreme learning machine with adaptive metrics of inputs

Knowledge-Based Systems
Weighted extreme learning machine for imbalance learning

Neurocomputing
Architecture selection for networks trained with extreme learning machine using localized generalization error model

Neurocomputing
Robust extreme learning machine

Neurocomputing
Regularized extreme learning machine for regression with missing data

Neurocomputing
Parallel extreme learning machine for regression based on MapReduce

Neurocomputing
Partitioned online sequential extreme learning machine for large ordered system modeling

Neurocomputing
Image classification based on effective extreme learning machine

Neurocomputing
Evolutionary extreme learning machine ensembles with size control

Neurocomputing
Feature selection for nonlinear models with extreme learning machines

Neurocomputing
EEG-based vigilance estimation using extreme learning machines

Neurocomputing
Extreme learning machine based genetic algorithm and its application in power system economic dispatch

Neurocomputing
Extreme learning machine based wind speed estimation and sensorless control for wind turbine power generation system

Neurocomputing
Optimizing extreme learning machines via ridge regression and batch intrinsic plasticity

Neurocomputing
Displacement prediction model of landslide based on ensemble of extreme learning machine

ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part IV
Echo state networks and extreme learning machines: a comparative study on seasonal streamflow series prediction

ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part II
A new optimization algorithm for single hidden layer feedforward neural networks11The third author would like to thank the support of RGC Grant PolyU. (5365/09E), NSFC Grant (10901170) and the Research Committee of the Hong Kong Polytechnic University.

Applied Soft Computing
A multi-objective micro genetic ELM algorithm

Neurocomputing
Point and prediction interval estimation for electricity markets with machine learning techniques and wavelet transforms

Neurocomputing
PCA-ELM: A Robust and Pruned Extreme Learning Machine Approach Based on Principal Component Analysis

Neural Processing Letters
Parallel Chaos Search Based Incremental Extreme Learning Machine

Neural Processing Letters
2013 Special Issue: Single-hidden-layer feed-forward quantum neural network based on Grover learning

Neural Networks
Extreme learning machine: a robust modeling technique? yes!

IWANN'13 Proceedings of the 12th international conference on Artificial Neural Networks: advances in computational intelligence - Volume Part I
Extending extreme learning machine with combination layer

IWANN'13 Proceedings of the 12th international conference on Artificial Neural Networks: advances in computational intelligence - Volume Part I
Generalized single-hidden layer feedforward networks

ISNN'13 Proceedings of the 10th international conference on Advances in Neural Networks - Volume Part I
Exploiting the relationships among several binary classifiers via data transformation

Pattern Recognition
Neural networks letter: Comments on the "No-Prop" algorithm

Neural Networks
Comparison of different approaches to visual terrain classification for outdoor mobile robots

Pattern Recognition Letters
Long-term time series prediction using OP-ELM

Neural Networks
Least Square Fast Learning Network for modeling the combustion efficiency of a 300WM coal-fired boiler

Neural Networks
A hierarchical structure of extreme learning machine (HELM) for high-dimensional datasets with noise

Neurocomputing
Review and performance comparison of SVM- and ELM-based classifiers

Neurocomputing
1-Norm extreme learning machine for regression and multiclass classification using Newton method

Neurocomputing
Parallelized extreme learning machine ensemble based on min-max modular network

Neurocomputing
Constructive multi-output extreme learning machine with application to large tanker motion dynamics identification

Neurocomputing
Meta-ELM: ELM with ELM hidden nodes

Neurocomputing
Clustering in extreme learning machine feature space

Neurocomputing
Fast sparse approximation of extreme learning machine

Neurocomputing
Double parallel feedforward neural network based on extreme learning machine with L 1/2 regularizer

Neurocomputing
Hierarchical extreme learning machine for feedforward neural network

Neurocomputing
Breast tumor detection in digital mammography based on extreme learning machine

Neurocomputing
Time-series processing of large scale remote sensing data with extreme learning machine

Neurocomputing
2-D defect profile reconstruction from ultrasonic guided wave signals based on QGA-kernelized ELM

Neurocomputing
Neural modeling of vapor compression refrigeration cycle with extreme learning machine

Neurocomputing
Predicting pupylation sites in prokaryotic proteins using pseudo-amino acid composition and extreme learning machine

Neurocomputing
Extreme learning machine towards dynamic model hypothesis in fish ethology research

Neurocomputing
Comparing the learning effectiveness of BP, ELM, I-ELM, and SVM for corporate credit ratings

Neurocomputing
Face Recognition and Micro-expression Recognition Based on Discriminant Tensor Subspace Analysis Plus Extreme Learning Machine

Neural Processing Letters
Online ship roll motion prediction based on grey sequential extreme learning machine

Neurocomputing
Learning of a single-hidden layer feedforward neural network using an optimized extreme learning machine

Neurocomputing
Genetic ensemble of extreme learning machine

Neurocomputing
Hybrid extreme rotation forest

Neural Networks
Learning to Rank with Extreme Learning Machine

Neural Processing Letters
Applications of Hybrid Extreme Rotation Forests for image segmentation

International Journal of Hybrid Intelligent Systems

Quantified Score

Hi-index	0.01

Visualization

Abstract

According to conventional neural network theories, single-hidden-layer feedforward networks (SLFNs) with additive or radial basis function (RBF) hidden nodes are universal approximators when all the parameters of the networks are allowed adjustable. However, as observed in most neural network implementations, tuning all the parameters of the networks may cause learning complicated and inefficient, and it may be difficult to train networks with nondifferential activation functions such as threshold networks. Unlike conventional neural network theories, this paper proves in an incremental constructive method that in order to let SLFNs work as universal approximators, one may simply randomly choose hidden nodes and then only need to adjust the output weights linking the hidden layer and the output layer. In such SLFNs implementations, the activation functions for additive nodes can be any bounded nonconstant piecewise continuous functions g:R→R and the activation functions for RBF nodes can be any integrable piecewise continuous functions g:R→R and ∫Rg(x)dx≠0. The proposed incremental method is efficient not only for SFLNs with continuous (including nondifferentiable) activation functions but also for SLFNs with piecewise continuous (such as threshold) activation functions. Compared to other popular methods such a new network is fully automatic and users need not intervene the learning process by manually tuning control parameters.