Classification methodologies of multilayer perceptrons with sigmoid activation functions

Authors:
Gao Daqi;Ji Yan
Affiliations:
Department of Computer Science, East China University of Science and Technology, Shanghai 200237, China and State Key Laboratory of Bioreactor Engineering, East China University of Science and Tec ...;Department of Computer Science, East China University of Science and Technology, Shanghai 200237, China
Venue:
Pattern Recognition
Year:
2005

Citing 16
Cited 7

Digital neural networks

Digital neural networks
Multilayer neural networks and Bayes decision theory

Neural Networks
Are Multilayer Perceptrons Adequate for Pattern Recognition and Verification?

IEEE Transactions on Pattern Analysis and Machine Intelligence
Derivation of the multilayer perception weight constraints for direct network interpretation and knowledge discovery

Neural Networks
Neural Networks for Pattern Recognition

Neural Networks for Pattern Recognition
On the capabilities of neural networks using limited precision weights

Neural Networks
Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)
Pattern classification using neural networks

IEEE Communications Magazine
Circular backpropagation networks for classification

IEEE Transactions on Neural Networks
Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions

IEEE Transactions on Neural Networks
A new neural network for cluster-detection-and-labeling

IEEE Transactions on Neural Networks
Reformulated radial basis neural networks trained by gradient descent

IEEE Transactions on Neural Networks
An overview of statistical learning theory

IEEE Transactions on Neural Networks
Classification ability of single hidden layer feedforward neural networks

IEEE Transactions on Neural Networks
Bounds on the number of hidden neurons in multilayer perceptrons

IEEE Transactions on Neural Networks
A novel training scheme for multilayered perceptrons to realize proper generalization and incremental learning

IEEE Transactions on Neural Networks

Task decomposition and modular single-hidden-layer perceptron classifiers for multi-class learning problems

Pattern Recognition
Bowel-sound pattern analysis using wavelets and neural networks with application to long-term, unsupervised, gastrointestinal motility monitoring

Expert Systems with Applications: An International Journal
Conquering the Needle-in-a-Haystack: How Correlated Input Variables Beneficially Alter the Fitness Landscape for Neural Networks

EvoBIO '09 Proceedings of the 7th European Conference on Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics
Collapse and reorganization patterns of social knowledge representation in evolving semantic networks

Information Sciences: an International Journal
Research of neural network classifier based on FCM and PSO for breast cancer classification

HAIS'12 Proceedings of the 7th international conference on Hybrid Artificial Intelligent Systems - Volume Part I
Accelerating FCM neural network classifier using graphics processing units with CUDA

Applied Intelligence
Global Artificial Bee Colony-Levenberq-Marquardt GABC-LM Algorithm for Classification

International Journal of Applied Evolutionary Computation

Quantified Score

Hi-index	0.01

Visualization

Abstract

This paper studies the classification mechanisms of multilayer perceptrons (MLPs) with sigmoid activation functions (SAFs). The viewpoint is presented that in the input space the hyperplanes determined by the hidden basis functions with values 0's do not play the role of decision boundaries, and such hyperplanes do not certainly go through the marginal regions between different classes. For solving an n-class problem, a single-hidden-layer perceptron with at least log2(n-1)=2 hidden nodes is needed. The final number of hidden neurons is still related to the sample distribution shapes and regions, but not to the number of samples and input dimensions. As a result, an empirical formula for optimally selecting the initial number of hidden nodes is proposed. The ranks of response matrixes of hidden layers should be taken as a main basis for pruning or growing the existing hidden neurons. A structure-fixed perceptron ought to learn more than one round from different starting weight points for one classification task, and only the group of weights and biases that has the best generalization performance should be reserved. Finally, three examples are given to verify the above viewpoints.