Keeping the neural networks simple by minimizing the description length of the weights
COLT '93 Proceedings of the sixth annual conference on Computational learning theory
Multi-Objective Optimization Using Evolutionary Algorithms
Multi-Objective Optimization Using Evolutionary Algorithms
Genetic Algorithms for Multiobjective Optimization: FormulationDiscussion and Generalization
Proceedings of the 5th International Conference on Genetic Algorithms
An overview of evolutionary algorithms in multiobjective optimization
Evolutionary Computation
A fast and elitist multiobjective genetic algorithm: NSGA-II
IEEE Transactions on Evolutionary Computation
The minimum description length principle in coding and modeling
IEEE Transactions on Information Theory
Mutation-based genetic neural network
IEEE Transactions on Neural Networks
| Hi-index | 0.00 |
Neural network design aims for high classification accuracy and low network architecture complexity. It is also known that simultaneous optimization of both model accuracy and complexity improves generalization while avoiding overfitting on data. We describe a neural network training procedure that uses multi-objective optimization to evolve networks which are optimal both with respect to classification accuracy and architectural complexity. The NSGA-II algorithm is employed to evolve a population of neural networks that are minimal in both training error and a Minimum Description Length-based network complexity measure. We further propose a pruning rule based on the following heuristic: connections to or from a node may be severed if their weight values are smaller than the network's smallest bias. Experiments on benchmark datasets show that the proposed evolutionary multi-objective approach to neural network design employing the bias-based pruning heuristic yields networks that have far fewer connections without seriously compromising generalization performance when compared to other existing evolutionary optimization algorithms.