Original Contribution: Multilayer feedforward networks with a nonpolynomial activation function can approximate any function

Authors:
Moshe Leshno;Vladimir Ya. Lin;Allan Pinkus;Shimon Schocken
Affiliations:
The Hebrew University, Israel;Technion, Israel;Technion, Israel;New York University, USA
Venue:
Neural Networks
Year:
1993

Citing 6
Cited 86

Multilayer feedforward networks are universal approximators

Neural Networks
On the approximate realization of continuous mappings by neural networks

Neural Networks
Connectionist learning procedures

Artificial Intelligence
Approximation capabilities of multilayer feedforward networks

Neural Networks
Approximation by ridge functions and neural networks with one hidden layer

Journal of Approximation Theory
Fundamentality of ridge functions

Journal of Approximation Theory

Towards robust model selection using estimation and approximation error bounds

COLT '96 Proceedings of the ninth annual conference on Computational learning theory
Nonlinear Versus Linear Learning Devices: A Procedural Perspective

Computational Economics
On the relevance of time in neural computation and learning

Theoretical Computer Science
Pareto-optimal formulations for cost versus colorimetric accuracy trade-offs in printer color management

ACM Transactions on Graphics (TOG)
Meta-Patterns and Higher Order Meta-Patterns in Cellular Systems

Artificial Intelligence Review
Fitting of a closed planar curve representing a profile of an archaeological fragment

Proceedings of the 2001 conference on Virtual reality, archeology, and cultural heritage
Approximating networks and extended Ritz method for the solution of functional optimization problems

Journal of Optimization Theory and Applications
Deep Teleology in Artificial Systems

Minds and Machines
Implementation of Kolmogorov Learning Algorithm for Feedforward Neural Networks

ICCS '01 Proceedings of the International Conference on Computational Science-Part II
An Efficient Hardware Implementation of Feed-Forward Neural Networks

Proceedings of the 14th International conference on Industrial and engineering applications of artificial intelligence and expert systems: engineering of intelligent systems
Extraction of Logical Rules from Data by Means of Piecewise-Linear Neural Networks

DS '02 Proceedings of the 5th International Conference on Discovery Science
On the complexity of computing and learning with multiplicative neural networks

Neural Computation
Analysis of Tikhonov regularization for function approximation by neural networks

Neural Networks
Zonal function network frames on the sphere

Neural Networks
Best approximation by linear combinations of characteristic functions of half-spaces

Journal of Approximation Theory
Approximation by neural networks with a bounded number of nodes at each level

Journal of Approximation Theory
An introduction to boosting and leveraging

Advanced lectures on machine learning
Greedy algorithms for classification—consistency, convergence rates, and adaptivity

The Journal of Machine Learning Research
Generalization error bounds for Bayesian mixture algorithms

The Journal of Machine Learning Research
Relaxed conditions for radial-basis function networks to be universal approximators

Neural Networks
On the Computational Power of Max-Min Propagation Neural Networks

Neural Processing Letters
An Efficient Hardware Implementation of Feed-Forward Neural Networks

Applied Intelligence
Functional multi-layer perceptron: a non-linear tool for functional data analysis

Neural Networks
Function approximation on non-Euclidean spaces

Neural Networks
A New On-Line Learning Model

Neural Computation
Neural Systems as Nonlinear Filters

Neural Computation
On the Computational Power of Winner-Take-All

Neural Computation
Almost Linear VC-Dimension Bounds for Piecewise Polynomial Networks

Neural Computation
Simultaneous Lp-approximation order for neural networks

Neural Networks
Neural network based optimal control of a biosynthesis process

International Journal of Knowledge-based and Intelligent Engineering Systems - Advanced Intelligent Techniques in Engineering Applications
An entropy-based quantum neuro-fuzzy inference system for classification applications

Neurocomputing
A new perspective for optimal portfolio selection with random fuzzy returns

Information Sciences: an International Journal
Letters: Convex incremental extreme learning machine

Neurocomputing
Minimization of error functionals over perceptron networks

Neural Computation
Incremental extreme learning machine with fully complex hidden nodes

Neurocomputing
The estimate for approximation error of neural networks: A constructive approach

Neurocomputing
Approximation of state-space trajectories by locally recurrent globally feed-forward neural networks

Neural Networks
Neural networks for optimal approximation of smooth and analytic functions

Neural Computation
A method for aerosol correction from the spectral variation in the visible and near infrared: application to the MERIS sensor

International Journal of Remote Sensing - MERIS (MEdium Resolution Imaging Spectrometer)
Enhanced random search based incremental extreme learning machine

Neurocomputing
Universal Approximation and QoS Violation Application of Extreme Learning Machine

Neural Processing Letters
Uniform Approximation Capabilities of Sum-of-Product and Sigma-Pi-Sigma Neural Networks

ISNN '07 Proceedings of the 4th international symposium on Neural Networks: Advances in Neural Networks
A Modified RBF Neural Network and Its Application in Radar

ISNN '07 Proceedings of the 4th international symposium on Neural Networks: Advances in Neural Networks, Part III
Evolutionary Combining of Basis Function Neural Networks for Classification

IWINAC '07 Proceedings of the 2nd international work-conference on The Interplay Between Natural and Artificial Computation, Part I: Bio-inspired Modeling of Cognitive Tasks
Learning Similarity Measures from Pairwise Constraints with Neural Networks

ICANN '08 Proceedings of the 18th international conference on Artificial Neural Networks, Part II
Simultaneous approximations of polynomials and derivatives and their applications to neural networks

Neural Computation
Convergence analysis of convex incremental neural networks

Annals of Mathematics and Artificial Intelligence
Stochastic sampling design using a multi-objective genetic algorithm and adaptive neural networks

Environmental Modelling & Software
Simplified neural networks algorithm for function approximation on discrete input spaces in high dimension-limited sample applications

Neurocomputing
Systemical convergence rate analysis of convex incremental feedforward neural networks

Neurocomputing
Approximation capabilities of multilayer fuzzy neural networks on the set of fuzzy-valued functions

Information Sciences: an International Journal
Neural Network Sieve Bootstrap Prediction Intervals: Some Real Data Evidence

Proceedings of the 2009 conference on New Directions in Neural Networks: 18th Italian Workshop on Neural Networks: WIRN 2008
The approximation operators with sigmoidal functions

Computers & Mathematics with Applications
An approximation by neural networkswith a fixed weight

Computers & Mathematics with Applications
Approximation Capability of Continuous Time Recurrent Neural Networks for Non-autonomous Dynamical Systems

ICANN '09 Proceedings of the 19th International Conference on Artificial Neural Networks: Part II
Neural networks with asymmetric activation function for function approximation

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
ANSER: adaptive neuron artificial neural network system for estimating rainfall

International Journal of Computers and Applications
The errors in simultaneous approximation by feed-forward neural networks

Neurocomputing
Neural network time series forecasting of finite-element mesh adaptation

Neurocomputing
Neural network prediction of the roll motion of a ship for intelligent course control

KES'07/WIRN'07 Proceedings of the 11th international conference, KES 2007 and XVII Italian workshop on neural networks conference on Knowledge-based intelligent information and engineering systems: Part III
New study on neural networks: the essential order of approximation

Neural Networks
Ensemble fractional sensitivity: a quantitative approach to neuron selection for decoding motor tasks

Computational Intelligence and Neuroscience - Special issue on signal processing for neural spike trains
Management of water resource systems in the presence of uncertainties by nonlinear approximation techniques and deterministic sampling

Computational Optimization and Applications
Constructive approximation to multivariate function by decay RBF neural network

IEEE Transactions on Neural Networks
Data mining using an adaptive HONN model with hyperbolic tangent neurons

PKAW'10 Proceedings of the 11th international conference on Knowledge management and acquisition for smart systems and services
Three-layer feedforward structures smoothly approximating polynomial functions

ICANN'10 Proceedings of the 20th international conference on Artificial neural networks: Part III
Approximation capability of interpolation neural networks

Neurocomputing
Multivariate sigmoidal neural network approximation

Neural Networks
The errors of simultaneous approximation of multivariate functions by neural networks

Computers & Mathematics with Applications
Essential rate for approximation by spherical neural networks

Neural Networks
Approximation of curves contained on the surface by freed-forward neural networks

AICI'11 Proceedings of the Third international conference on Artificial intelligence and computational intelligence - Volume Part III
Learning from pairwise constraints by Similarity Neural Networks

Neural Networks
Numerical assessment of metamodelling strategies in computationally intensive optimization

Environmental Modelling & Software
Stable hybrid control based on discrete-event automata and receding-horizon neural regulators

Automatica (Journal of IFAC)
The errors of approximation for feedforward neural networks in the Lp metric

Mathematical and Computer Modelling: An International Journal
Approximation of level continuous fuzzy-valued functions by multilayer regular fuzzy neural networks

Mathematical and Computer Modelling: An International Journal
Application of radial basis function and generalized regression neural networks in non-linear utility function specification for travel mode choice modelling

Mathematical and Computer Modelling: An International Journal
Use of artificial neural network simulation metamodelling to assess groundwater contamination in a road project

Mathematical and Computer Modelling: An International Journal
Approximation properties of local bases assembled from neural network transfer functions

Mathematical and Computer Modelling: An International Journal
Boosting GARCH and neural networks for the prediction of heteroskedastic time series

Mathematical and Computer Modelling: An International Journal
Nonlinearity in Forecasting of High-Frequency Stock Returns

Computational Economics
Comparing studies of learning methods for human face gender recognition

CCBR'12 Proceedings of the 7th Chinese conference on Biometric Recognition
Optimization of the weights and asymmetric activation function family of neural network for time series forecasting

Expert Systems with Applications: An International Journal
The universal approximation capabilities of mellin approximate identity neural networks

ISNN'13 Proceedings of the 10th international conference on Advances in Neural Networks - Volume Part I
Review and performance comparison of SVM- and ELM-based classifiers

Neurocomputing
Fast decorrelated neural network ensembles with random weights

Information Sciences: an International Journal

Quantified Score

Hi-index	0.01

Visualization

Abstract

Several researchers characterized the activation function under which multilayer feedforward networks can act as universal approximators. We show that most of all the characterizations that were reported thus far in the literature are special cases of the following general result: A standard multilayer feedforward network with a locally bounded piecewise continuous activation function can approximate any continuous function to any degree of accuracy if and only if the network's activation function is not a polynomial. We also emphasize the important role of the threshold, asserting that without it the last theorem does not hold.