Learning internal representations by error propagation
Parallel distributed processing: explorations in the microstructure of cognition, vol. 1
Training connectionist networks with queries and selective sampling
Advances in neural information processing systems 2
Generalizing from case studies: a case study
ML92 Proceedings of the ninth international workshop on Machine learning
Original Contribution: Stacked generalization
Neural Networks
An introduction to Kolmogorov complexity and its applications
An introduction to Kolmogorov complexity and its applications
Implementing inner drive through competence reflection
Proceedings of the second international conference on From animals to animats 2 : simulation of adaptive behavior: simulation of adaptive behavior
The nature of statistical learning theory
The nature of statistical learning theory
Machine learning, neural and statistical classification
Evaluation and Selection of Biases in Machine Learning
Machine Learning - Special issue on bias evaluation and selection
Machine Learning
Combination of Multiple Classifiers Using Local Accuracy Estimates
IEEE Transactions on Pattern Analysis and Machine Intelligence
A decision-theoretic generalization of on-line learning and an application to boosting
Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
Machine Learning - Special issue on inductive transfer
Algorithmic stability and sanity-check bounds for leave-one-out cross-validation
COLT '97 Proceedings of the tenth annual conference on Computational learning theory
Combining labeled and unlabeled data with co-training
COLT' 98 Proceedings of the eleventh annual conference on Computational learning theory
A survey of connectionist network reuse through transfer
Learning to learn
An introduction to support Vector Machines: and other kernel-based learning methods
An introduction to support Vector Machines: and other kernel-based learning methods
Machine Learning
A perspective view and survey of meta-learning
Artificial Intelligence Review
Fast Parzen Density Estimation Using Clustering-Based Branch and Bound
IEEE Transactions on Pattern Analysis and Machine Intelligence
Reliable Classifications with Machine Learning
ECML '02 Proceedings of the 13th European Conference on Machine Learning
Characterization of Classification Algorithms
EPIA '95 Proceedings of the 7th Portuguese Conference on Artificial Intelligence: Progress in Artificial Intelligence
Ridge Regression Confidence Machine
ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Enhancing Supervised Learning with Unlabeled Data
ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Learning to Predict the Leave-One-Out Error of Kernel Based Classifiers
ICANN '01 Proceedings of the International Conference on Artificial Neural Networks
Data perturbation for escaping local maxima in learning
Eighteenth national conference on Artificial intelligence
Simple Principles of Metalearning
Simple Principles of Metalearning
The Journal of Machine Learning Research
Constructive Incremental Learning from Only Local Information
Neural Computation
Generalized Additive Models (Texts in Statistical Science)
Generalized Additive Models (Texts in Statistical Science)
Estimation of individual prediction reliability using the local sensitivity analysis
Applied Intelligence
Comparison of approaches for estimating reliability of individual regression predictions
Data & Knowledge Engineering
Machine Learning and Data Mining: Introduction to Principles and Algorithms
Machine Learning and Data Mining: Introduction to Principles and Algorithms
Transduction with confidence and credibility
IJCAI'99 Proceedings of the 16th international joint conference on Artificial intelligence - Volume 2
LIBSVM: A library for support vector machines
ACM Transactions on Intelligent Systems and Technology (TIST)
UAI'98 Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence
| Hi-index | 0.00 |
In machine learning and its risk-sensitive applications (e.g. medicine, engineering, business), the reliability estimates for individual predictions provide more information about the individual prediction error (the difference between the true label and regression prediction) than the average accuracy of predictive model (e.g. relative mean squared error). Furthermore, they enable the users to distinguish between more and less reliable predictions. The empirical evaluations of the existing individual reliability estimates revealed that the successful estimates’ performance depends on the used regression model and on the particular problem domain. In the current paper, we focus on that problem as such and propose and empirically evaluate two approaches for automatic selection of the most appropriate estimate for a given domain and regression model: the internal cross-validation approach and the meta-learning approach. The testing results of both approaches demonstrated an advantage in the performance of dynamically chosen reliability estimates to the performance of the individual reliability estimates. The best results were achieved using the internal cross-validation procedure, where reliability estimates significantly positively correlated with the prediction error in 73% of experiments. In addition, the preliminary testing of the proposed methodology on a medical domain demonstrated the potential for its usage in practice.