Differential Evolution Training Algorithm for Feed-Forward Neural Networks

Authors:
Jarmo Ilonen;Joni-Kristian Kamarainen;Jouni Lampinen
Affiliations:
Laboratory of Information Processing, Lappeenranta University of Technology, P.O. Box 20, FIN-53851 Lappeenranta, Finland. e-mail: ilonen@lut.fi;Laboratory of Information Processing, Lappeenranta University of Technology, P.O. Box 20, FIN-53851 Lappeenranta, Finland. e-mail: jkamarai@lut.fi;Laboratory of Information Processing, Lappeenranta University of Technology, P.O. Box 20, FIN-53851 Lappeenranta, Finland.
Venue:
Neural Processing Letters
Year:
2003

Citing 3
Cited 52

Combinatorial evolution of regression nodes in feedforward neural networks

Neural Networks
Mechanical engineering design optimization by differential evolution

New ideas in optimization
Differential Evolution – A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces

Journal of Global Optimization

Incorporating directional information within a differential evolution algorithm for multi-objective optimization

Proceedings of the 8th annual conference on Genetic and evolutionary computation
Shape recognition based on neural networks trained by differential evolution algorithm

Neurocomputing
2007 Special Issue: Nonlinear analog predictor analysis: A coupled neural network/analog model for climate downscaling

Neural Networks
Prediction of cutting forces in turning process using de-neural networks

AIAP'07 Proceedings of the 25th conference on Proceedings of the 25th IASTED International Multi-Conference: artificial intelligence and applications
Learning algorithm of wavelet network based on sampling theory

Neurocomputing
Differential Evolution and Levenberg Marquardt Trained Neural Network Scheme for Nonlinear System Identification

Neural Processing Letters
An effective hybrid DE-based algorithm for multi-objective flow shop scheduling with limited buffers

Computers and Operations Research
Threshold accepting trained principal component neural network and feature subset selection: Application to bankruptcy prediction in banks

Applied Soft Computing
A novel differential evolution algorithm for bi-criteria no-wait flow shop scheduling problems

Computers and Operations Research
A DE-based approach to no-wait flow-shop scheduling

Computers and Industrial Engineering
A novel hybrid discrete differential evolution algorithm for blocking flow shop scheduling problems

Computers and Operations Research
A soft computing-based approach for integrated training and rule extraction from artificial neural networks: DIFACONN-miner

Applied Soft Computing
Differential evolution algorithm with strategy adaptation for global numerical optimization

IEEE Transactions on Evolutionary Computation
Rule extraction from differential evolution trained radial basis function network using genetic algorithms

CASE'09 Proceedings of the fifth annual IEEE international conference on Automation science and engineering
A differential mutation operator for the archive population of multi-objective evolutionary algorithms

CEC'09 Proceedings of the Eleventh conference on Congress on Evolutionary Computation
Differential migration: sensitivity analysis and comparison study

CEC'09 Proceedings of the Eleventh conference on Congress on Evolutionary Computation
Multi-units Unified Process Optimization Under Uncertainty Based on Differential Evolution with Hypothesis Test

ICIC '07 Proceedings of the 3rd International Conference on Intelligent Computing: Advanced Intelligent Computing Theories and Applications. With Aspects of Artificial Intelligence
Differential evolution trained radial basis function network: application to bankruptcy prediction in banks

International Journal of Bio-Inspired Computation
Sensitivity versus accuracy in multiclass problems using memetic Pareto evolutionary neural networks

IEEE Transactions on Neural Networks
A Lamarckian Hybrid of Differential Evolution and Conjugate Gradients for Neural Network Training

Neural Processing Letters
A differential evolution algorithm with self-adapting strategy and control parameters

Computers and Operations Research
A differential evolution based neural network approach to nonlinear system identification

Applied Soft Computing
Differential evolution algorithm with ensemble of parameters and mutation strategies

Applied Soft Computing
Design of artificial neural networks using differential evolution algorithm

ICONIP'10 Proceedings of the 17th international conference on Neural information processing: models and applications - Volume Part II
Seeker optimization algorithm for tuning the structure and parameters of neural networks

Neurocomputing
Evolving artificial neural networks using adaptive differential evolution

IBERAMIA'10 Proceedings of the 12th Ibero-American conference on Advances in artificial intelligence
Nonlinear system identification using memetic differential evolution trained neural networks

Neurocomputing
The effects of scheduling, workload type and consolidation scenarios on virtual machine performance and their prediction through optimized artificial neural networks

Journal of Systems and Software
RankDE: learning a ranking function for information retrieval using differential evolution

Proceedings of the 13th annual conference on Genetic and evolutionary computation
Evolving neural networks: a comparison between differential evolution and particle swarm optimization

ICSI'11 Proceedings of the Second international conference on Advances in swarm intelligence - Volume Part I
Weighting Efficient Accuracy and Minimum Sensitivity for Evolving Multi-Class Classifiers

Neural Processing Letters
Comparison of evolutionary computation techniques for noise injected neural network training to estimate longitudinal dispersion coefficients in rivers

Expert Systems with Applications: An International Journal
Enhancing the search ability of differential evolution through orthogonal crossover

Information Sciences: an International Journal
Training neural networks with harmony search algorithms for classification problems

Engineering Applications of Artificial Intelligence
Multi-sensor based prediction of metal deposition in pulsed gas metal arc welding using various soft computing models

Applied Soft Computing
Solving rotated multi-objective optimization problems using differential evolution

AI'04 Proceedings of the 17th Australian joint conference on Advances in Artificial Intelligence
Evolutionary learning using a sensitivity-accuracy approach for classification

HAIS'10 Proceedings of the 5th international conference on Hybrid Artificial Intelligence Systems - Volume Part II
A multi-objective neural network based method for cover crop identification from remote sensed data

Expert Systems with Applications: An International Journal
An informative differential evolution with self adaptive re-clustering technique

SEMCCO'11 Proceedings of the Second international conference on Swarm, Evolutionary, and Memetic Computing - Volume Part I
A differential evolution algorithm for lot-streaming flow shop scheduling problem

ICIC'11 Proceedings of the 7th international conference on Advanced Intelligent Computing
Global hybrid ant bee colony algorithm for training artificial neural networks

ICCSA'12 Proceedings of the 12th international conference on Computational Science and Its Applications - Volume Part I
A hybrid evolving and gradient strategy for approximating policy evaluation on online critic-actor learning

ISNN'12 Proceedings of the 9th international conference on Advances in Neural Networks - Volume Part I
An Improved Decision Support System for Detection of Lesions in Mammograms Using Differential Evolution Optimized Wavelet Neural Network

Journal of Medical Systems
Optimization methodology based on neural networks and self-adaptive differential evolution algorithm applied to an aerobic fermentation process

Applied Soft Computing
Differential evolution trained kernel principal component WNN and kernel binary quantile regression: Application to banking

Knowledge-Based Systems
Differential Evolution for automatic rule extraction from medical databases

Applied Soft Computing
G-HABC Algorithm for Training Artificial Neural Networks

International Journal of Applied Metaheuristic Computing
Performance of informative differential evolution algorithm with self adaptive re-clustering technique on the problems of electromagnetism: the linear array antenna synthesis

SEMCCO'12 Proceedings of the Third international conference on Swarm, Evolutionary, and Memetic Computing
Global artificial bee colony algorithm for boolean function classification

ACIIDS'13 Proceedings of the 5th Asian conference on Intelligent Information and Database Systems - Volume Part I
Flexible job shop scheduling using hybrid differential evolution algorithms

Computers and Industrial Engineering
A new back-propagation neural network optimized with cuckoo search algorithm

ICCSA'13 Proceedings of the 13th international conference on Computational Science and Its Applications - Volume 1
Comparing large number of metaheuristics for artificial neural networks training to predict water temperature in a natural river

Computers & Geosciences

Quantified Score

Hi-index	0.00

Visualization

Abstract

An evolutionary optimization method over continuous search spaces, differential evolution, has recently been successfully applied to real world and artificial optimization problems and proposed also for neural network training. However, differential evolution has not been comprehensively studied in the context of training neural network weights, i.e., how useful is differential evolution in finding the global optimum for expense of convergence speed. In this study, differential evolution has been analyzed as a candidate global optimization method for feed-forward neural networks. In comparison to gradient based methods, differential evolution seems not to provide any distinct advantage in terms of learning rate or solution quality. Differential evolution can rather be used in validation of reached optima and in the development of regularization terms and non-conventional transfer functions that do not necessarily provide gradient information.