Neural networks and the bias/variance dilemma
Neural Computation
Improving regression estimation: Averaging methods for variance reduction with extensions to general convex measure optimization
Negative correlation learning and evolutionary design of neural network ensembles
Negative correlation learning and evolutionary design of neural network ensembles
Linear combiners for classifier fusion: some theoretical and experimental results
MCS'03 Proceedings of the 4th international conference on Multiple classifier systems
Between two extremes: examining decompositions of the ensemble objective function
MCS'05 Proceedings of the 6th international conference on Multiple Classifier Systems
Evolutionary ensembles with negative correlation learning
IEEE Transactions on Evolutionary Computation
Local averaging of heterogeneous regression models
International Journal of Hybrid Intelligent Systems
Non-strict heterogeneous Stacking
Pattern Recognition Letters
A theoretical framework for multiple neural network systems
Neurocomputing
Greedy regression ensemble selection: Theory and an application to water quality prediction
Information Sciences: an International Journal
Meta-Modeling in Multiobjective Optimization
Multiobjective Optimization
Evolving an Ensemble of Neural Networks Using Artificial Immune Systems
SEAL '08 Proceedings of the 7th International Conference on Simulated Evolution and Learning
Negative correlation in incremental learning
Natural Computing: an international journal
Modeling UCS as a mixture of experts
Proceedings of the 11th Annual conference on Genetic and evolutionary computation
Random Ordinality Ensembles$\colon$ A Novel Ensemble Method for Multi-valued Categorical Data
MCS '09 Proceedings of the 8th International Workshop on Multiple Classifier Systems
Profiling of Mass Spectrometry Data for Ovarian Cancer Detection Using Negative Correlation Learning
ICANN '09 Proceedings of the 19th International Conference on Artificial Neural Networks: Part II
Evolutionary Ensemble for In Silico Prediction of Ames Test Mutagenicity
ICIC '07 Proceedings of the 3rd International Conference on Intelligent Computing: Advanced Intelligent Computing Theories and Applications. With Aspects of Artificial Intelligence
Diversity exploration and negative correlation learning on imbalanced data sets
IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
An Anticorrelation Kernel for Subsystem Training in Multiple Classifier Systems
The Journal of Machine Learning Research
An improved random subspace method and its application to EEG signal classification
MCS'07 Proceedings of the 7th international conference on Multiple classifier systems
Ensemble learning in linearly combined classifiers via negative correlation
MCS'07 Proceedings of the 7th international conference on Multiple classifier systems
An empirical study of multilayer perceptron ensembles for regression tasks
IEA/AIE'10 Proceedings of the 23rd international conference on Industrial engineering and other applications of applied intelligent systems - Volume Part II
A randomized model ensemble approach for reconstructing signals from faulty sensors
Expert Systems with Applications: An International Journal
A novel multi-view learning developed from single-view patterns
Pattern Recognition
Clustering students to generate an ensemble to improve standard test score predictions
AIED'11 Proceedings of the 15th international conference on Artificial intelligence in education
A D-GMDH model for time series forecasting
Expert Systems with Applications: An International Journal
Making Diversity Enhancement Based on Multiple Classifier System by Weight Tuning
Neural Processing Letters
Between two extremes: examining decompositions of the ensemble objective function
MCS'05 Proceedings of the 6th international conference on Multiple Classifier Systems
Relationship between diversity and correlation in multi-classifier systems
PAKDD'10 Proceedings of the 14th Pacific-Asia conference on Advances in Knowledge Discovery and Data Mining - Volume Part II
Training regression ensembles by sequential target correction and resampling
Information Sciences: an International Journal
Tomographic considerations in ensemble bias/variance decomposition
MCS'10 Proceedings of the 9th international conference on Multiple Classifier Systems
“Good” and “bad” diversity in majority vote ensembles
MCS'10 Proceedings of the 9th international conference on Multiple Classifier Systems
Expert pruning based on genetic algorithm in regression problems
ACIIDS'12 Proceedings of the 4th Asian conference on Intelligent Information and Database Systems - Volume Part III
Neural network ensembles to determine growth multi-classes in predictive microbiology
HAIS'12 Proceedings of the 7th international conference on Hybrid Artificial Intelligent Systems - Volume Part II
Ensemble approaches for regression: A survey
ACM Computing Surveys (CSUR)
ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part V
An investigation into the application of ensemble learning for entailment classification
Information Processing and Management: an International Journal
A survey of multiple classifier systems as hybrid systems
Information Fusion
Fast decorrelated neural network ensembles with random weights
Information Sciences: an International Journal
Hi-index | 0.00 |
Ensembles are a widely used and effective technique in machine learning---their success is commonly attributed to the degree of disagreement, or 'diversity', within the ensemble. For ensembles where the individual estimators output crisp class labels, this 'diversity' is not well understood and remains an open research issue. For ensembles of regression estimators, the diversity can be exactly formulated in terms of the covariance between individual estimator outputs, and the optimum level is expressed in terms of a bias-variance-covariance trade-off. Despite this, most approaches to learning ensembles use heuristics to encourage the right degree of diversity. In this work we show how to explicitly control diversity through the error function. The first contribution of this paper is to show that by taking the combination mechanism for the ensemble into account we can derive an error function for each individual that balances ensemble diversity with individual accuracy. We show the relationship between this error function and an existing algorithm called negative correlation learning, which uses a heuristic penalty term added to the mean squared error function. It is demonstrated that these methods control the bias-variance-covariance trade-off systematically, and can be utilised with any estimator capable of minimising a quadratic error function, for example MLPs, or RBF networks. As a second contribution, we derive a strict upper bound on the coefficient of the penalty term, which holds for any estimator that can be cast in a generalised linear regression framework, with mild assumptions on the basis functions. Finally we present the results of an empirical study, showing significant improvements over simple ensemble learning, and finding that this technique is competitive with a variety of methods, including boosting, bagging, mixtures of experts, and Gaussian processes, on a number of tasks.