Optimal Linear Combination of Neural Networks for Improving Classification Performance

Authors:
Naonori Ueda
Affiliations:
NTT Communications Science Lab, Kyoto, Japan
Venue:
IEEE Transactions on Pattern Analysis and Machine Intelligence
Year:
2000

Citing 9
Cited 44

Original Contribution: Stacked generalization

Neural Networks
Improving regression estimation: Averaging methods for variance reduction with extensions to general convex measure optimization

Improving regression estimation: Averaging methods for variance reduction with extensions to general convex measure optimization
Stacked regressions

Machine Learning
Optimal linear combinations of neural networks

Neural Networks
Partial BFGS update and efficient step-length calculation for three-layer neural networks

Neural Computation
Neural Networks for Pattern Recognition

Neural Networks for Pattern Recognition
Neural Network Ensembles

IEEE Transactions on Pattern Analysis and Machine Intelligence
Discriminative metric design for robust pattern recognition

IEEE Transactions on Signal Processing
Parallel consensual neural networks

IEEE Transactions on Neural Networks

On Fusers that Perform Better than Best Sensor

IEEE Transactions on Pattern Analysis and Machine Intelligence
Ensembling neural networks: many could be better than all

Artificial Intelligence
Lithology Recognition by Neural Network Ensembles

SBIA '02 Proceedings of the 16th Brazilian Symposium on Artificial Intelligence: Advances in Artificial Intelligence
Error Rejection in Linearly Combined Multiple Classifiers

MCS '01 Proceedings of the Second International Workshop on Multiple Classifier Systems
Analysis of Linear and Order Statistics Combiners for Fusion of Imbalanced Classifiers

MCS '02 Proceedings of the Third International Workshop on Multiple Classifier Systems
Performance Analysis and Comparison of Linear Combiners for Classifier Fusion

Proceedings of the Joint IAPR International Workshop on Structural, Syntactic, and Statistical Pattern Recognition
Extracting symbolic rules from trained neural network ensembles

AI Communications - Special issue on Artificial intelligence advances in China
A Theoretical and Experimental Analysis of Linear Combiners for Multiple Classifier Systems

IEEE Transactions on Pattern Analysis and Machine Intelligence
Effective classification of noisy data streams with attribute-oriented dynamic classifier selection

Knowledge and Information Systems
Extracting symbolic rules from trained neural network ensembles

AI Communications - Artificial Intelligence Advances in China
Class-dependent PCA, MDC and LDA: A combined classifier for pattern classification

Pattern Recognition
Adaptive fusion and co-operative training for classifier ensembles

Pattern Recognition
Trainable fusion rules. I. Large sample size case

Neural Networks
Calligraphic Interfaces: Classifier combination for sketch-based 3D part retrieval

Computers and Graphics
Classifier ensemble selection using hybrid genetic algorithms

Pattern Recognition Letters
Estimation and decision fusion: A survey

Neurocomputing
Increasing classification efficiency with multiple mirror classifiers

Expert Systems with Applications: An International Journal
Multistage RBF neural network ensemble learning for exchange rates forecasting

Neurocomputing
Resampling-based selective clustering ensembles

Pattern Recognition Letters
A neural-network-based nonlinear metamodeling approach to financial time series forecasting

Applied Soft Computing
Incremental construction of classifier and discriminant ensembles

Information Sciences: an International Journal
Data dependency in multiple classifier systems

Pattern Recognition
Cluster Ensembles Based on Vector Space Embeddings of Graphs

MCS '09 Proceedings of the 8th International Workshop on Multiple Classifier Systems
A boundary based classifier combination method

CCDC'09 Proceedings of the 21st annual international conference on Chinese control and decision conference
Classifier combination based on confidence transformation

Pattern Recognition
Combining fingerprint and hand-geometry verification decisions

AVBPA'03 Proceedings of the 4th international conference on Audio- and video-based biometric person authentication
Data dependence in combining classifiers

MCS'03 Proceedings of the 4th international conference on Multiple classifier systems
Neural net ensembles for lithology recognition

MCS'03 Proceedings of the 4th international conference on Multiple classifier systems
A PSO-based weighting method for linear combination of neural networks

Computers and Electrical Engineering
Sparse ensembles using weighted combination methods based on linear programming

Pattern Recognition
A reliability-based RBF network ensemble model for foreign exchange rates predication

ICONIP'06 Proceedings of the 13th international conference on Neural information processing - Volume Part III
Partial AUC maximization in a linear combination of dichotomizers

Pattern Recognition
Dealing with limited data in ballistic impact scenarios: an empirical comparison of different neural network approaches

Applied Intelligence
A probability-based unified 3d shape search

SAMT'06 Proceedings of the First international conference on Semantic and Digital Media Technologies
Recognition of complex human behaviors in pool environment using foreground silhouette

ISVC'05 Proceedings of the First international conference on Advances in Visual Computing
Decoding rules for error correcting output code ensembles

MCS'05 Proceedings of the 6th international conference on Multiple Classifier Systems
An optimization methodology for machine learning strategies and regression problems in ballistic impact scenarios

Applied Intelligence
Classifiers selection in ensembles using genetic algorithms for bankruptcy prediction

Expert Systems with Applications: An International Journal
2012 Special Issue: Multi-column deep neural network for traffic sign classification

Neural Networks
Sketch-based 3D engineering part class browsing and retrieval

SBM'06 Proceedings of the Third Eurographics conference on Sketch-Based Interfaces and Modeling
Reduced analytical dependency modeling for classifier fusion

ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part III
Linear classifier combination and selection using group sparse regularization and hinge loss

Pattern Recognition Letters
Multiscale convolutional neural networks for vision: based classification of cells

ACCV'12 Proceedings of the 11th Asian conference on Computer Vision - Volume Part II
Depression recognition based on dynamic facial and vocal expression features using partial least square regression

Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge

Quantified Score

Hi-index	0.15

Visualization

Abstract

With a focus on classification problems, this paper presents a new method for linearly combining multiple neural network classifiers based on statistical pattern recognition theory. In our approach, several neural networks are first selected based on which works best for each class in terms of minimizing classification errors. Then, they are linearly combined to form an ideal classifier that exploits the strengths of the individual classifiers. In this approach, the minimum classification error (MCE) criterion is utilized to estimate the optimal linear weights. In this formulation, because the classification decision rule is incorporated into the cost function, a more suitable better combination of weights for the classification objective could be obtained. Experimental results using artificial and real data sets show that the proposed method can construct a better combined classifier that outperforms the best single classifier in terms of overall classification errors for test data.