Training a sigmoidal node is hard
Neural Computation
On finding global optima for the hinge fitting problem
Computers and Operations Research
The combination limit in multimedia retrieval
MULTIMEDIA '03 Proceedings of the eleventh ACM international conference on Multimedia
Vibration Control of Block Forming Machine Based on an Artificial Neural Network
ISNN '07 Proceedings of the 4th international symposium on Neural Networks: Advances in Neural Networks
A neural network of smooth hinge functions
IEEE Transactions on Neural Networks
Nonlinear regression with piecewise affine models based on RBFN
ICANN'05 Proceedings of the 15th international conference on Artificial neural networks: formal models and their applications - Volume Part II
An improved three-term optical backpropagation algorithm
International Journal of Artificial Intelligence and Soft Computing
Identification of piecewise affine systems via mixed-integer programming
Automatica (Journal of IFAC)
| Hi-index | 0.01 |
This paper presents a computationally efficient algorithm for function approximation with piecewise linear sigmoidal nodes. A one hidden layer network is constructed one node at a time using the well-known method of fitting the residual. The task of fitting an individual node is accomplished using a new algorithm that searches for the best fit by solving a sequence of quadratic programming problems. This approach offers significant advantages over derivative-based search algorithms (e.g., backpropagation and its extensions). Unique characteristics of this algorithm include: finite step convergence, a simple stopping criterion, solutions that are independent of initial conditions, good scaling properties and a robust numerical implementation. Empirical results are included to illustrate these characteristics