Dynamics of learning near singularities in radial basis function networks

Authors:
Haikun Wei;Shun-ichi Amari
Affiliations:
Amari Research Unit, RIKEN Brain Science Institute, Saitama, 3510198, Japan and School of Automation, Southeast University, Nanjing, 210096, China and Department of Brain Science and Engineering, ...;Amari Research Unit, RIKEN Brain Science Institute, Saitama, 3510198, Japan
Venue:
Neural Networks
Year:
2008

Citing 6
Cited 6

A regularity condition of the information matrix of a multilayer perceptron network

Neural Networks
Online learning in radial basis function networks

Neural Computation
Local minima and plateaus in hierarchical structures of multilayer perceptions

Neural Networks
Singularities Affect Dynamics of Learning in Neuromanifolds

Neural Computation
Dynamics of learning near singularities in layered networks

Neural Computation
Dynamics of Learning in Multilayer Perceptrons Near Singularities

IEEE Transactions on Neural Networks

Dynamics of learning near singularities in layered networks

Neural Computation
A study on the use of imputation methods for experimentation with Radial Basis Function Network classifiers handling missing attribute values: The good synergy between RBFNs and EventCovering method

Neural Networks
Conditional mixture model for correlated neuronal spikes

Neural Computation
Learning Non-linear Multivariate Dynamics of Motion in Robotic Manipulators

International Journal of Robotics Research
Parameter estimation of q-Gaussian Radial Basis Functions Neural Networks with a Hybrid Algorithm for binary classification

Neurocomputing
Dreaming of mathematical neuroscience for half a century

Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

The radial basis function (RBF) networks are one of the most widely used models for function approximation in the regression problem. In the learning paradigm, the best approximation is recursively or iteratively searched for based on observed data (teacher signals). One encounters difficulties in such a process when two component basis functions become identical, or when the magnitude of one component becomes null. In this case, the number of the components reduces by one, and then the reduced component recovers as the learning process proceeds further, provided such a component is necessary for the best approximation. Strange behaviors, especially the plateau phenomena, have been observed in dynamics of learning when such reduction occurs. There exist singularities in the space of parameters, and the above reduction takes place at the singular regions. This paper focuses on a detailed analysis of the dynamical behaviors of learning near the overlap and elimination singularities in RBF networks, based on the averaged learning equation that is applicable to both on-line and batch mode learning. We analyze the stability on the overlap singularity by solving the eigenvalues of the Hessian explicitly. Based on the stability analysis, we plot the analytical dynamic vector fields near the singularity, which are then compared to those real trajectories obtained by a numeric method. We also confirm the existence of the plateaus in both batch and on-line learning by simulation.