Activation functions defined on higher-dimensional spaces for approximation on compact sets with and without scaling

Authors:
Yoshifusa Ito
Affiliations:
Aichi-Gakuin University, Nisshin-shi, Aichi-ken, 470-0195 Japan
Venue:
Neural Computation
Year:
2003

Citing 13
Cited 0

Real and complex analysis, 3rd ed.

Real and complex analysis, 3rd ed.
Multilayer feedforward networks are universal approximators

Neural Networks
On the approximate realization of continuous mappings by neural networks

Neural Networks
Representation of functions by superpositions of a step or sigmoid function and their applications to neural network theory

Neural Networks
Universal approximation using radial-basis-function networks

Neural Computation
Approximation of functions on a compact set by finite sums of a sigmoid function without scaling

Neural Networks
Original Contribution: Approximation of continuous functions on Rd by linear combinations of shifted rotations of a sigmoid function with and without scaling

Neural Networks
Kolmogorov's theorem and multilayer neural networks

Neural Networks
Original Contribution: Uniqueness of the weights for minimal feedforward nets with a given input-output map

Neural Networks
Approximation and radial-basis-function networks

Neural Computation
Nonlinear approximations using elliptic basis function networks

Circuits, Systems, and Signal Processing
Universal approximation using feedforward neural networks: a survey of some existing methods, and some new results

Neural Networks
A Weak Condition on Linear Independence of Unscaled Shifts of a Function and Finite Mappings by Neural Networks

ICANN '02 Proceedings of the International Conference on Artificial Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

Let g be a slowly increasing function of locally bounded variation defined on Rc, 1 ≤ c ≤ d. We investigate when g can be an activation function of the hidden-layer units of three-layer neural networks that approximate continuous functions on compact sets. If the support of the Fourier transform of g includes a converging sequence of points with distinct distances from the origin, it can be an activation function without scaling. If and only if the support of its Fourier transform includes a point other than the origin, it can be an activation function with scaling. We also look for a condition on which an activation function can be used for approximation without rotation. Any nonpolynomial functions can be activation functions with scaling, and many familiar functions, such as sigmoid functions and radial basis functions, can be activation functions without scaling. With or without scaling, some of them defined on Rd can be used without rotation even if they are not spherically symmetric.