Black Box Test Case Prioritization for Regression Testing Using Bayesian Networks

Abstract

In testing, regression testing can be defined as the re-execution of all test cases previously executed to rule out the fact that some functionalities that were working previously have been broken by the newly introduced fixes or system changes. The impact is adverse if the software system involves money or life critical systems. Constraints like time and resources cannot allow re-execution of the whole collection of test cases that were executed previously and due to security reasons system codebase is never availed to the testing team and if it is availed, the testing team might not be technically competent to extract value from the code. The study investigated existing implementations of test case prioritization and had incorporated machine learning algorithm as part of the implementation. The study implemented a black box test case prioritization model using Bayesian Networks, after which a prototype was developed to prioritize test cases using the model. The study also validated that the developed prototype is effective in prioritizing test cases. Based on the methodology selected for the study, data was collected from public dataset, Kaggle all the variables under study were available in the data. Analysis was done on the data to visualize distribution of data. Feature engineering was done to the features to improve the performance of the model. The model was implemented using the established correlations between dependent variable; likelihood of detecting bugs against the independent variables; complexity value of the developed system, the level of experience for the developer who participated in the development of the system under test, the change history of the system, the bug history of the system and the tester assessment for likelihood of detecting bugs based on experience .A prototype was developed and tests done to validate the effectiveness, simulations were also carried out using the test data. The model was evaluated to establish its effectiveness in prioritizing test cases. The Bayesian Networks model performed slightly better in classification accuracy and confusion matrix when compared to Gaussian Naïve Bayes and Support Vector Machine respectively. The study achieved the set-out objectives by carrying out systematic literature review on previous work and identifying the gap in regression testing for black box test case prioritization(Catal & Mishra, 2012), a model was implemented, and a prototype developed to deal with the issue of black box test case prioritization for regression testing. The effectiveness of the developed model was evaluated against other models and BN model was slightly better than the other models. The study achieved its objectives, with the proposed solution software development teams will be able to prioritize test cases without the need to access source code using minimum training data. This will ensure high quality software is released and reduce the risk of defect leakage which can cause harm or threaten lives.