Measuring the Impacts Individual Process Maturity Attributes Have on Software Products

• Authors: Frank McGarry;Steve Burke;Bill Decker
• Venue: METRICS '98 Proceedings of the 5th International Symposium on Software Metrics
• Year: 1998

For over 20 years, the System Engineering and Analysis Support (SEAS) Center has been working with NASA in the development of Mission Operations and Data Systems software. In support of and in collaboration with NASA, SEAS has participated in numerous studies and activities designed to measure and analyze the quality of software products as well as the technologies and processes used to produce those products. This paper describes the approach, activities, data, and early results of studies attempting to more generally determine the impact that software processes have on the end item software products. Individual software process attributes as well as software process improvement as a whole will be analyzed. The paper describes the approach to identifying project data for both the product and process and describes one aspect of the series of studies being carried out; the results of analyzing the relationship between project quality and a corresponding process rating derived from the Software Engineering Institute (SEI) in its Capability Maturity Model (CMM). The studies have taken place within the CSC/SEAS Center where software systems are developed and maintained in support of NASA flight projects. The projects have ranged in size from 10 KDSI (thousand delivered source instructions) to over 700 KDSI, and the applications have specifically been for mission support activities, which include systems for control center operations, data processing activities, command and control and flight dynamics disciplines. In order to carry out this analysis, over 90 software projects were analyzed where information was available characterizing both the end software product as well as the methods and general processes used to produce that product. Defect data, effort, cycle time, and size were some of the product measures examined. CMM Key Process Area (KPA) data from Software Capability Evaluations (SCEs) and Software Process Assessments (SPAs), In-Progress Process Audits (IPPAs), Flight Dynamics Subjective Evaluation Forms (SEFs), and project history reports were some of the measures and information examined representing the process of the software. This particular study analyzed the potential impact that the CMM Maturity Level 2 and 3 KPAs had on product defect rates, productivity, cycle time, and effort variance. Also analyzed was how these four product measures changed over time. The study showed:1.There was not a significant correlation between quantified process maturity using our derived process measure and the four product measures (for those projects with detailed CMM data). 2.Software productivity and software defect rates improved consistently over the 14-year period, independent of software process activities based on CMM improvements. 3. Software development cycle time and software effort estimation improved significantly for the SEAS Center after the start of process improvement activities based on CMM (for one class of systems).