SlideShare a Scribd company logo

A Validation of Object-Oriented Design Metrics as Quality Indicators

•
•
V
vie_dels

The document summarizes a research paper that empirically validated several object-oriented design metrics proposed by Chidamber and Kemerer as indicators of fault-prone classes. The study analyzed 6 metrics on 180 classes from a system. Univariate analysis found 5 metrics to be significantly correlated with fault probability. Multivariate analysis using these 5 metrics achieved better prediction of faulty classes than models using traditional code metrics. The research validated that these OO design metrics can help identify fault-prone classes early in the development lifecycle.

1 of 46
A Validation of Object-Oriented Design Metrics as Quality Indicators Evi Yulianti 1006833110 Iis Solichah 1006800094 Mubarik Ahmad 1006833294  Victor R. Basili  Lionel C. Briand  Walcelio L. Melo University of Mariland
Content Article & Author Introduction Data Analysis Case Study Conclusion & Future Work
Article Publication:  Journal IEEE Transactions on Software Engineering Issue Date: Oct 1996 Volume: 22 Issue:10 On page(s): 751 - 761
Author Dr. Victor R. Basili (University of Maryland) Department of Computer Science , Professor, 1970 – Present Institute for Advanced Computer Studies , Research Professor, 1984-Present Dr. Lionel C. Briand (Carleton University) Canada Research Chair (Tier I) in Software Quality Engineering Dr. Walcelio (Walt) L. Melo, Professor, Catholic University of Brasilia, DF, Brazil , 1997-2001 Lead Architect, Model Driven Solutions, 2008-now
Introduction
Introduction Time & resource consuming activity help manager : 1. make decisions, plan and schedule activities, 2. allocate resources for the different software activities identify fault-prone modules TESTING SOFTWARE METRIC … ?
Introduction (cont’) Metrics must be defined and validated in order to be used in industry Empirical validation aims at demonstrating the usefulness of a measure in practice and is, therefore, a crucial activity to establish the overall validity of a measure. ability to identify fault-prone classes
Chidamber & Kemerer’s metric [13] Weighted Methods per Class (WMC) Depth of Inheritance Tree of a class (DIT) Number Of Children of a Class (NOC) Coupling Between Object classes (CBO) Response For a Class (RFC) Lack of Cohesion on Methods (LCOM)
Hypothesis H-WMC H-DIT H-NOC H-CBO H-RFC H-LCOM
Data Analysis
DATA ANALYSIS Assess empirically whether the OO design metrics defined in [13] are useful predictors or fault-prone classes. Used analysis: Descriptive distributions of the OO Metrics Univariate and multivariate analysis of the relationship between OO Metrics and fault-proneness.
Distribution and Correlation Analysis Distribution of the analyzed OO metrics based on 180 presented classes. Number of Class
Distribution and Correlation Analysis (cont.) Descriptive statistics of the metric distributions.
Distribution and Correlation Analysis (cont.) Linear Pearson's correlations (R': Coefficient of determination) between the studied OO metrics are very weak. These metrics are mostly statistically independent.
The Relationships Between Fault Probability and OO Metrics Analysis Methodology Explanatory variable: metrics from [13] Response variable: was a fault detected in a class during testing phases? (binary) Used methods: Logistic regression Univariate Multivariate
The Relationships Between Fault Probability and OO Metrics (cont.) Resulting Table
The Relationships Between Fault Probability and OO Metrics (cont.) Statistics: Coefficient: the estimated regression coefficient.  Statistical significance (p-value)
Univariate Analysis A nalyze the relationships between six OO metrics introduced in [ 13 ] and the probability of fault detection in a class during test phases. Test the hypothesis.
Univariate Analysis (cont.) Test Result: (analyzed from Table 3) WMC : H-WMC is supported. DIT : H-DIT is supported. RFC : H-RFC is supported. NOC : H-NOC is not supported. LCOM : can’t be analyzed. CBO : H-CBO is supported.
Multivariate Analysis T o evaluate the predictive capability of those metrics that had been assessed sufficiently significant in the univariate analysis. O nly the metrics that significantly improve the predictive power of the multivariate model
Result 1 The figures before parentheses i n the right column are the number of classes class i f i ed as faulty The figures within the parentheses are the faults contained i n those classes. Based on OO Design Metrics [13] Using such a model for classification, the results are obtained by using a classification threshold of π (Fault detection) = 0.5, i.e., when π > 0.5, the class is classified as faulty and, otherwise, as nonfaulty.
Result II Based on Code Metrics [2] 112 classes (predicted as faulty) out of 180 would be inspected and 51 faulty classes out of 58 would be detected
Result III Accuracies of OO and Code Metrics C orrectness (percentage of classes correctly predicted as faulted) C ompleteness(percentage of faulty classes detected) Values between parentheses present predictions correctness and completeness when classes are weighted according to number of faults they contain
CASE STUDY: DATA ANALYSIS FOR SAMPLE CLASSES
Class Diagram
Distribution Analysis Distribution of the analyzed OO metrics based on eight sample classes from “Tugas 1”.
Distribution Analysis (cont.) Distribution of the analyzed OO metrics based on eight sample classes from “Tugas 1”.
Distribution Analysis (cont.) Descriptive statistics of the metric value distributions. WMC DIT RFC NOC LCOM CBO Maximum 15 2 18 2 93 2 Minimum 1 0 1 0 0 0 Mean 6.875 0.5 10.375 0.375 11.625 1.375
Logistic Regression It is used for prediction of the probability of occurrence of an event It makes use of several predictor variables Multivariate logistic regression function: Univariate logistic regression is special case where only one variable appears
SPSS V.15 VARIABLE VIEW
DATA VIEW
Univariate Analysis:
coefficient constant z = - 0,513 + 0,075*WMC R 2 Odds ratio π = exp(z) / (1+exp(z))
WMC z= -0,513 + 0,075*WMC π = exp(z) / (1+exp(z)) Class WMC π Tool.java 1 0,392218 CTextbox.java 1 0,392218 DrawingPackage.java 3 0,428494 Screen.java 5 0,465555 CCircle.java 8 0,521736 ShapeList.java 10 0,558974 CRect.java 12 0,59556 CShape.java 15 0,648397
DIT z= -1,386 + 21,566*DIT π = exp(z) / (1+exp(z)) Class DIT π Tool.java 0 0,200047 Screen.java 0 0,200047 ShapeList.java 0 0,200047 CShape.java 0 0,200047 DrawingPackage.java 0 0,200047 CCircle.java 1 1 CRect.java 1 1 CTextbox.java 2 1
NOC z= 0,196 - 0,5309*NOC π = exp(z) / (1+exp(z)) Class NOC π Tool.java 0 0,548844 Screen.java 0 0,548844 ShapeList.java 0 0,548844 CCircle.java 0 0,548844 CTextbox.java 0 0,548844 DrawingPackage.java 0 0,548844 CRect.java 1 0,417049 CShape.java 2 0,296129
CBO z= -2,884 – 2,027*CBO π = exp(z) / (1+exp(z)) Class CBO π Screen.java 0 0,05295 Tool.java 1 0,297967 ShapeList.java 1 0,297967 CShape.java 1 0,297967 CCircle.java 2 0,763145 CRect.java 2 0,763145 CTextbox.java 2 0,763145 DrawingPackage.java 2 0,763145
RFC z= -0,941 + 0,09*RFC π = exp(z) / (1+exp(z)) Class RFC π Tool.java 1 0,299223 Screen.java 5 0,379658 CTextbox.java 6 0,401072 CCircle.java 9 0,467297 CRect.java 14 0,579081 CShape.java 15 0,600848 DrawingPackage.java 15 0,600848 ShapeList.java 18 0,663515
LCOM z= 0,288 - 0,231*LCOM π = exp(z) / (1+exp(z)) Class LCOM π Tool.java 0 0,571506429 Screen.java 0 0,571506429 ShapeList.java 0 0,571506429 CCircle.java 0 0,571506429 CRect.java 0 0,571506429 CTextbox.java 0 0,571506429 DrawingPackage.java 0 0,571506429 CShape.java 93 6,23919E-10
Multivariate Analysis:
constant coefficient WMC coefficient DIT coefficient NOC coefficient CBO coefficient RFC R 2 z= 50,930 - 4,249*WMC - 33,403*DIT + 28,433*NOC + 5,256*CBO -1,885*RFC π = exp(z) / (1+exp(z))
Related Works
CONCLUSION & FUTURE WORK
Conclusions F ive out of the six Chidamber and Kemerer’s OO metrics appear to be useful to predict class fault-proneness during the high- and low-level design phases of the life-cycle Chidamber and Kemerer’s OO metrics show to be better predictors than the best set of ”traditional” code metrics, which can only be collected during later phases of the software development processes. M ost of these metrics appear to be complementary indicators which are relatively independent from each other
Future Works Replicating this study in an industrial setting: A sample of large-scale projects developed in C++ and Ida95 in the framework of the NAsA Goddard Flight Dynamics Division (Software Engineering Laboratory). Studying the variations, in terms of metric definitions and experimental results, between different OO programming languages. Extending the empirical investigation to other OO metrics proposed in the literature and develop improved metrics
Paper Reference List [2] Amadeus Software Research, Getting Started With Amadeus, Amadeus Measurement System, 1994. [13] S.R. Chidamber and C.F. Kemerer, “A Metrics Suite for Object-Oriented Design,” IEEE Trans. Software Eng., vol. 20, no. 6, pp. 476493, June 1994.

More Related Content

A Validation of Object-Oriented Design Metrics as Quality Indicators

  • 1. A Validation of Object-Oriented Design Metrics as Quality Indicators Evi Yulianti 1006833110 Iis Solichah 1006800094 Mubarik Ahmad 1006833294  Victor R. Basili  Lionel C. Briand  Walcelio L. Melo University of Mariland
  • 2. Content Article & Author Introduction Data Analysis Case Study Conclusion & Future Work
  • 3. Article Publication:  Journal IEEE Transactions on Software Engineering Issue Date: Oct 1996 Volume: 22 Issue:10 On page(s): 751 - 761
  • 4. Author Dr. Victor R. Basili (University of Maryland) Department of Computer Science , Professor, 1970 – Present Institute for Advanced Computer Studies , Research Professor, 1984-Present Dr. Lionel C. Briand (Carleton University) Canada Research Chair (Tier I) in Software Quality Engineering Dr. Walcelio (Walt) L. Melo, Professor, Catholic University of Brasilia, DF, Brazil , 1997-2001 Lead Architect, Model Driven Solutions, 2008-now
  • 6. Introduction Time & resource consuming activity help manager : 1. make decisions, plan and schedule activities, 2. allocate resources for the different software activities identify fault-prone modules TESTING SOFTWARE METRIC … ?
  • 7. Introduction (cont’) Metrics must be defined and validated in order to be used in industry Empirical validation aims at demonstrating the usefulness of a measure in practice and is, therefore, a crucial activity to establish the overall validity of a measure. ability to identify fault-prone classes
  • 8. Chidamber & Kemerer’s metric [13] Weighted Methods per Class (WMC) Depth of Inheritance Tree of a class (DIT) Number Of Children of a Class (NOC) Coupling Between Object classes (CBO) Response For a Class (RFC) Lack of Cohesion on Methods (LCOM)
  • 9. Hypothesis H-WMC H-DIT H-NOC H-CBO H-RFC H-LCOM
  • 11. DATA ANALYSIS Assess empirically whether the OO design metrics defined in [13] are useful predictors or fault-prone classes. Used analysis: Descriptive distributions of the OO Metrics Univariate and multivariate analysis of the relationship between OO Metrics and fault-proneness.
  • 12. Distribution and Correlation Analysis Distribution of the analyzed OO metrics based on 180 presented classes. Number of Class
  • 13. Distribution and Correlation Analysis (cont.) Descriptive statistics of the metric distributions.
  • 14. Distribution and Correlation Analysis (cont.) Linear Pearson's correlations (R': Coefficient of determination) between the studied OO metrics are very weak. These metrics are mostly statistically independent.
  • 15. The Relationships Between Fault Probability and OO Metrics Analysis Methodology Explanatory variable: metrics from [13] Response variable: was a fault detected in a class during testing phases? (binary) Used methods: Logistic regression Univariate Multivariate
  • 16. The Relationships Between Fault Probability and OO Metrics (cont.) Resulting Table
  • 17. The Relationships Between Fault Probability and OO Metrics (cont.) Statistics: Coefficient: the estimated regression coefficient.  Statistical significance (p-value)
  • 18. Univariate Analysis A nalyze the relationships between six OO metrics introduced in [ 13 ] and the probability of fault detection in a class during test phases. Test the hypothesis.
  • 19. Univariate Analysis (cont.) Test Result: (analyzed from Table 3) WMC : H-WMC is supported. DIT : H-DIT is supported. RFC : H-RFC is supported. NOC : H-NOC is not supported. LCOM : can’t be analyzed. CBO : H-CBO is supported.
  • 20. Multivariate Analysis T o evaluate the predictive capability of those metrics that had been assessed sufficiently significant in the univariate analysis. O nly the metrics that significantly improve the predictive power of the multivariate model
  • 21. Result 1 The figures before parentheses i n the right column are the number of classes class i f i ed as faulty The figures within the parentheses are the faults contained i n those classes. Based on OO Design Metrics [13] Using such a model for classification, the results are obtained by using a classification threshold of π (Fault detection) = 0.5, i.e., when π > 0.5, the class is classified as faulty and, otherwise, as nonfaulty.
  • 22. Result II Based on Code Metrics [2] 112 classes (predicted as faulty) out of 180 would be inspected and 51 faulty classes out of 58 would be detected
  • 23. Result III Accuracies of OO and Code Metrics C orrectness (percentage of classes correctly predicted as faulted) C ompleteness(percentage of faulty classes detected) Values between parentheses present predictions correctness and completeness when classes are weighted according to number of faults they contain
  • 24. CASE STUDY: DATA ANALYSIS FOR SAMPLE CLASSES
  • 26. Distribution Analysis Distribution of the analyzed OO metrics based on eight sample classes from “Tugas 1”.
  • 27. Distribution Analysis (cont.) Distribution of the analyzed OO metrics based on eight sample classes from “Tugas 1”.
  • 28. Distribution Analysis (cont.) Descriptive statistics of the metric value distributions. WMC DIT RFC NOC LCOM CBO Maximum 15 2 18 2 93 2 Minimum 1 0 1 0 0 0 Mean 6.875 0.5 10.375 0.375 11.625 1.375
  • 29. Logistic Regression It is used for prediction of the probability of occurrence of an event It makes use of several predictor variables Multivariate logistic regression function: Univariate logistic regression is special case where only one variable appears
  • 30. SPSS V.15 VARIABLE VIEW
  • 33. coefficient constant z = - 0,513 + 0,075*WMC R 2 Odds ratio π = exp(z) / (1+exp(z))
  • 34. WMC z= -0,513 + 0,075*WMC π = exp(z) / (1+exp(z)) Class WMC π Tool.java 1 0,392218 CTextbox.java 1 0,392218 DrawingPackage.java 3 0,428494 Screen.java 5 0,465555 CCircle.java 8 0,521736 ShapeList.java 10 0,558974 CRect.java 12 0,59556 CShape.java 15 0,648397
  • 35. DIT z= -1,386 + 21,566*DIT π = exp(z) / (1+exp(z)) Class DIT π Tool.java 0 0,200047 Screen.java 0 0,200047 ShapeList.java 0 0,200047 CShape.java 0 0,200047 DrawingPackage.java 0 0,200047 CCircle.java 1 1 CRect.java 1 1 CTextbox.java 2 1
  • 36. NOC z= 0,196 - 0,5309*NOC π = exp(z) / (1+exp(z)) Class NOC π Tool.java 0 0,548844 Screen.java 0 0,548844 ShapeList.java 0 0,548844 CCircle.java 0 0,548844 CTextbox.java 0 0,548844 DrawingPackage.java 0 0,548844 CRect.java 1 0,417049 CShape.java 2 0,296129
  • 37. CBO z= -2,884 – 2,027*CBO π = exp(z) / (1+exp(z)) Class CBO π Screen.java 0 0,05295 Tool.java 1 0,297967 ShapeList.java 1 0,297967 CShape.java 1 0,297967 CCircle.java 2 0,763145 CRect.java 2 0,763145 CTextbox.java 2 0,763145 DrawingPackage.java 2 0,763145
  • 38. RFC z= -0,941 + 0,09*RFC π = exp(z) / (1+exp(z)) Class RFC π Tool.java 1 0,299223 Screen.java 5 0,379658 CTextbox.java 6 0,401072 CCircle.java 9 0,467297 CRect.java 14 0,579081 CShape.java 15 0,600848 DrawingPackage.java 15 0,600848 ShapeList.java 18 0,663515
  • 39. LCOM z= 0,288 - 0,231*LCOM π = exp(z) / (1+exp(z)) Class LCOM π Tool.java 0 0,571506429 Screen.java 0 0,571506429 ShapeList.java 0 0,571506429 CCircle.java 0 0,571506429 CRect.java 0 0,571506429 CTextbox.java 0 0,571506429 DrawingPackage.java 0 0,571506429 CShape.java 93 6,23919E-10
  • 41. constant coefficient WMC coefficient DIT coefficient NOC coefficient CBO coefficient RFC R 2 z= 50,930 - 4,249*WMC - 33,403*DIT + 28,433*NOC + 5,256*CBO -1,885*RFC π = exp(z) / (1+exp(z))
  • 44. Conclusions F ive out of the six Chidamber and Kemerer’s OO metrics appear to be useful to predict class fault-proneness during the high- and low-level design phases of the life-cycle Chidamber and Kemerer’s OO metrics show to be better predictors than the best set of ”traditional” code metrics, which can only be collected during later phases of the software development processes. M ost of these metrics appear to be complementary indicators which are relatively independent from each other
  • 45. Future Works Replicating this study in an industrial setting: A sample of large-scale projects developed in C++ and Ida95 in the framework of the NAsA Goddard Flight Dynamics Division (Software Engineering Laboratory). Studying the variations, in terms of metric definitions and experimental results, between different OO programming languages. Extending the empirical investigation to other OO metrics proposed in the literature and develop improved metrics
  • 46. Paper Reference List [2] Amadeus Software Research, Getting Started With Amadeus, Amadeus Measurement System, 1994. [13] S.R. Chidamber and C.F. Kemerer, “A Metrics Suite for Object-Oriented Design,” IEEE Trans. Software Eng., vol. 20, no. 6, pp. 476493, June 1994.

Editor's Notes

  1. Tabel 1 menunjukkan bahwa class-class yang diobservasi rata-rata memiliki: -DIT (kedalaman inheritance) rendah NOC rendah (rata-rata class hanya memiliki sedikit children) LCOM juga rendah (rata-rata class memiliki high-cohesion) Dari data tersebut, dapat dilihat bahwa Metrics tersebut tidak dapat melakukan differentiate terhadap sample classes.
  2. Korelasi antar metric rendah. Hanya yang dicetak tebal yang memiliki korelasi cukup signifikan. Pada scatterplots: relationship antara CBO dan RFC tidak disebabkan oleh outliers.
  3. Logistic regression: a standard technique based on maximum likelihood estimation, to analyze the relationships between metrics and the faultproneness of classes. Univariate -> to evaluate the relationship of each of the metrics in isolation and faultproneness Multivariate -> to evaluate the predictive capability of those metrics that had been assessed sufficiently significant in the univariate analysis Formula di atas adalah persamaan relasi untuk multivariate logistic regression. Univariate adalah salah satu kasus khusus dari multivariate (ketika hanya satu variable yang muncul dalam persamaan). Phi menyatakan probabilitas ditemukannya fault pada class saat validasi. Xi menyatakan design metrics sebagai explanatory variable pada model. ( covaviates of the logistic regression equation )
  4. Data statistik yang digunakan pada tabel 3 dan 4: coefficient: semakin besar koefisien, maka pengaruh explanatory variable terhadap respons variable semakin besar.  : selisih odd-ratio -> menunjukkan penambahan/pengurangan odd ratio ketika X bertambah satu unit.  (X): menunjukkan pengaruh metrics terhadap variable yang diprediksi. p-value: keakuratan estimasi koefisien (selisih koefisien)
  5. Penjelasan NOC: Kebanyakan class dalam sample tidak memiliki lebih dari satu anak. Reuse merupakan faktor signifikan negatif terhadap fault-density [5]. Large NOC are less fault-prone. Penjelasan LCOM: LCOM tidak dapat dianalisis untuk memprediksi nilai probabilitas fault, karena nilai-nilai yang seharusnya negatif dalam LCOM direset menjadi nol. Hal ini mengakibatkan tidak dapat dilakukannya perbandingan yang fair antar class.
  6. Dari total 58 class yang pada actual case memiliki fault, kita bisa memprediksikan 48 class faulty. Dari total 268 fault yang pada actual case terjadi, kita bisa memprediksikan 250 fault. To summarize, results show that the studied OO metrics are useful predictors of fault-proneness.
  7. Code metric ini disediakan oleh Amadeus tool [2]. Mendeteksi 112 class sebagai faulty (actual faulty: 58 class).  Terdapat 61 class yang harus diperiksa, padahal bukan faulty.
  8. in logistic regression R2 > 0,3 is considered good so our model seems to be a good predictor.