An Empirical Comparison of Ideal and Empirical ROC-Based Reject Rules
MLDM '07 Proceedings of the 5th international conference on Machine Learning and Data Mining in Pattern Recognition
| Hi-index | 0.00 |
When there are uncertainties in pattern recognition it may be better to introduce a rejection option to reduce the total costs, and two rules have been proposed: Chow's rule based on posterior probabilities and Tortorella's rule based on ROC curves. However, both have shortcomings for the application in practice: First, it is extremely difficult to obtain the exact posterior probability for each example to be recognized; Second, for small data size, the associated ROC curves may have very little number of convex points, resulting in the ineffectiveness of Tortorella's rule. This paper proposes a new bootstrap algorithm for obtaining sampling distributions of test example scores produced by Fisher LDA. These distributions can not only convert the scores into posterior probabilities but also generate a ROC curve with a lot of convex points. Thus, this bootstrap method can improve the effectiveness of Chow's rule and Tortorella's rule in the real applications.