Finding software metrics threshold values using ROC curves
Journal of Software Maintenance and Evolution: Research and Practice
| Hi-index | 0.01 |
Previous research has shown that object-oriented software metrics can predict error proneness and maintenance activities of classes, as well as project progress during the development process. However, whether the metrics' predictive capability still exists after the software is released is not known. This research has three objectives: (1) to investigate whether software metrics can predict class error proneness; (2) to investigate whether software metrics can predict the class error probability in three error-severity categories; and (3) to identify the distinct metrics threshold values that are clearly associated with error categories, by examining the data collected from the post-release evolution of an open-source system. Our empirical research is conducted on three releases of the Eclipse project---a continuously evolving, large industrial-strength open source system. The research results show that some object-oriented metrics can predict class error probabilities in the three error-severity categories. Furthermore, we report that the classes can be grouped into different error categories based on the probability-predication models. By using the large open-source system, we also verify the previous research result: some OO metrics can predict class error proneness. In previous research and practice, threshold values of object-oriented metrics were identified. However, most metric threshold values used in practice, such as the threshold values suggested by the Software Assurance Technology Center at the National Aeronautic and Space Agency (NASA), were suggested intuitively. Few empirical studies have identified threshold values of object-oriented metrics from empirical error data. Our research reports that there are significant differences among the metric means in the three error-severity categories and these means can be used to calculate the metrics threshold values.