A Study on Class Imbalancing Feature Selection and Ensembles on Software Reliability Prediction

On the benefit of feature selection and ensemble feature selection for fuzzy k-nearest neighbor classification

Software defect prediction models are used for predicting high risk software components. Feature selection has significant impact on the prediction performance of the software defect prediction models since redundant and unimportant features make the prediction model more difficult to learn. Ensemble feature selection has recently emerged as a new methodology for enhancing feature selection performance. This paper proposes a new multi-criteria-decision-making (MCDM) based ensemble feature selection (EFS) method. This new method is termed as MCDM-EFS. The proposed method, MCDM-EFS, first generates the decision matrix signifying the feature’s importance score with respect to various existing feature selection methods. Next, the decision matrix is used as the input to well-known MCDM method TOPSIS for assigning a final rank to each feature. The proposed approach is validated by an experimental study for predicting software defects using two classifiers K-nearest neighbor (KNN) and naïve bayes (NB) over five open-source datasets. The predictive performance of the proposed approach is compared with existing feature selection algorithms. Two evaluation metrics – nMCC and G-measure are used to compare predictive performance. The experimental results show that the MCDM-EFS significantly improves the predictive performance of software defect prediction models against other feature selection methods in terms of nMCC as well as G-measure.

Software reliability is one of the key aspects of software quality estimation and prediction during software testing period. Hence, accurate prediction of software reliability is an important but critical job. Machine Learning (ML) techniques have been proven victorious in providing superior results than traditional techniques for software reliability prediction. Generally, ML models entail sufficient data for training to achieve improved generalization. Inadequate training data may lead to land at suboptimal solution. This article suggests a method of enriching training dataset through exploration and incorporation of virtual data from existing data. For boost up the overall accuracy and reducing the risk of model selection an ensemble framework of five ML methods is suggested. The extended software reliability dataset is then exposed to the constituent models as well as the ensemble approach separately to estimate the future data. Extensive simulation results on a couple of software reliability datasets reveal that our proposed model significantly improves the prediction accuracy.

A hybrid ensemble feature selection design for candidate biomarkers discovery from transcriptome profiles

Size, complexity, and human dependency on software--based products have grown dramatically during past decades. Software developers are struggling to deliver reliable software with acceptable level of quality, within given budget and schedule. One measure of software quality and reliability is the number of residual faults. Therefore, researchers are focusing on the identification of the number of fault presents in the software or identification of program modules that are most likely to contain faults. A lot of models have been developed using various techniques. A common approach is followed for software reliability prediction utilizing failure data. Software reliability and quality prediction is highly desired by the stakeholders, developers, managers, and end users. Detecting software faults early during development will definitely improve the reliability and quality in cost-effective way.KeywordsSoftware ReliabilityFault PredictionSoftware MetricsCapability Maturity ModelTest Case PrioritizationThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

BackgroundFeature selection methods aim at identifying a subset of features that improve the prediction performance of subsequent classification models and thereby also simplify their interpretability. Preceding studies demonstrated that single feature selection methods can have specific biases, whereas an ensemble feature selection has the advantage to alleviate and compensate for these biases.ResultsThe software EFS (Ensemble Feature Selection) makes use of multiple feature selection methods and combines their normalized outputs to a quantitative ensemble importance. Currently, eight different feature selection methods have been integrated in EFS, which can be used separately or combined in an ensemble.ConclusionEFS identifies relevant features while compensating specific biases of single methods due to an ensemble approach. Thereby, EFS can improve the prediction accuracy and interpretability in subsequent binary classification models.AvailabilityEFS can be downloaded as an R-package from CRAN or used via a web application at http://EFS.heiderlab.de.

In this paper we consider non-parametric estimation methods for software reliability assessment without specifying the fault distribution, where the underlying stochastic process to describe software fault-counts in the system testing is given by a non-homogeneous Poisson process. A comprehensive approach based on the kernel estimation is provided with several kernel functions and bandwidth estimations. Next, we develop interval estimation methods via the non-parametric bootstrap, and derive the confidence regions of several reliability measures such as the expected cumulative number of software faults, software intensity function, quantitative software reliability as well. The resulting data-driven methodology can give the useful probabilistic information on the software reliability prediction under the incomplete knowledge on fault distribution. In illustrative examples with a real software fault data, it is shown that the proposed methods provide useful software reliability measures under uncertainty from the view point of frequentist analysis.

In reality, the fault detection (or correction) phenomenon and software reliability estimation in the operational phase are different from that in the testing phase. The fault removal continues at a slower rate during the operational phase. In this paper, we will investigate some techniques for software reliability prediction and measurement in the operational phase. We first review how some software reliability growth models (SRGMs) based on non-homogeneous Poisson processes (NHPPs) can be readily derived based on a unified theory for NHPP models. Under this general framework, we can not only verify some conventional SRGMs but also derive some new SRGMs that can be used for software reliability measurement in the operational phase. That is, based on the unified theory, we can incorporate the concept of multiple changepoints into software reliability modeling. We can formularize and simulate the fault detection process in the operational phase. Some numerical illustrations based on real software failure data are also presented. The experimental results show that the proposed models can easily reflect the possible changes of fault detection process and offer quantitative analysis on software failure behavior in field operation.

Over the past 30 years, many software reliability growth models (SRGMs) have been proposed for estimation of reliability growth of software. In fact, effective debugging will not be easy because the fault may not be immediately obvious. In the past, some researchers ever used an infinite server queueing (ISQ) model to describe the software debugging behavior. An infinite-server queueing model is considered where access of customers to service is controlled by a gate and the gate is open only if all servers are free. However, the finite server queueing (FSQ) model is first advantageously modeled as an infinite-server system. Thus, in this paper, we will show how to incorporate both FSQ and ISQ models into software reliability estimation and prediction. In addition, here we also consider the factor of perfect/imperfect debugging. Experimental results show that the proposed framework to incorporate both fault detection and correction processes for SRGM has a fairly accurate prediction capability.

Ensemble gene (feature) selection is a promising new strategy with many benefits including more stable gene lists and improved classification results. The ensemble portion is achieved through multiple runs of feature selection which are then aggregated into a single result. The critical question is how many iterations of feature selection are appropriate. Too few iterations can make classification performance suffer. However, too many iterations will cause issues regarding computational costs. The goal is to choose the correct number of iterations to maximize classification performance without expending too much computational power. Our paper is an in-depth study on the effect of the number of iterations of feature selection on classification performance. Our work employs eleven DNA microarray datasets, on which we apply various ensemble methods, feature selection techniques, classifiers, and feature subset sizes. The results show that using 10 iterations of feature ranking during ensemble feature selection is not sufficient to optimize classification results and that a larger number of iterations is required (20 or 50). However, there is very little distinction between 20 iterations and 50 iterations, as both produce very similar classification results. Our recommendation is to use 20 iterations because while 20 iterations and 50 iterations perform similarly, 20 iterations has a much smaller computation time. To our knowledge there has not been a previous study as expansive as this one on the effects of the number of iterations of feature selection on ensemble feature selection.

This paper presents a Bayes nonparametric approach for tracking and predicting software reliability. We use the common assumptions on the software operational environment to get a stochastic model where the successive times between software failures are exponentially distributed; their failure rates have Markov priors. Under these general assumptions we give Bayes estimates of the parameters that assess and predict the software reliability. We give algorithms (based on Monte-Carlo methods) to compute these Bayes estimates. Our approach allows the reliability analyst to construct a personal software reliability model simply by specifying the available prior knowledge; afterwards the results in this paper can be used to get Bayes estimates of the useful reliability parameters. Examples of possible prior physical knowledge concerning the software testing and correction environments are given. The maximum-entropy principle is used to translate this knowledge to prior distributions on the failure-rate process. Our approach is used to study some simulated and real failure data sets.

In this thesis, we present methodologies involving a data structure called the debugging graph whereby the predictive performance of software reliability models can be analyzed and improved under laboratory conditions. This procedure substitutes the averages of large sample sets for the single point samples normally used as inputs to these models and thus supports scrutiny of their performances with less random input data.\nInitially, we describe the construction of an extensive database of empirical reliability data which we derived by testing each partially debugged version of subject software represented by complete or partial debugging graphs. We demonstrate how these data can be used to assign relative sizes to known bugs and to simulate multiple debugging sessions. We then present the results from a series of proof-of-concept experiments.\nWe show that controlling fault recovery order as represented by the data input to some well-known reliability models can enable them to produce more accurate predictions and can mitigate anomalous effects we attribute to manifestations of the fault interaction phenomenon. Since limited testing resources are common in the real world, we demonstrate the use of two approximation techniques, the surrogate oracle and path truncations, to render the application of our methodologies computationally feasible outside a laboratory setting. We report results which support the assertion that reliability data collected from just a partial debugging graph and subject to these approximations qualitatively agrees with those collected under ideal laboratory conditions, provided one accounts for optimistic bias introduced by the surrogate in later prediction stages. We outline an algorithmic approach for using data derived from a partial debugging graph to improve software reliability predictions, and show its complexity to be no worse than O(n2). We summarize some outstanding questions as areas for future investigations of and improvements to the software reliability prediction process.

Software defect detection is a critical research topic in the field of software engineering, aiming to identify potential defects during the development process to improve software quality and reduce maintenance costs. This study proposes a novel feature selection and defect prediction classification algorithm based on a multi-strategy enhanced Parrot Optimization (PO) algorithm. Firstly, to address the limitations of the original Parrot Optimization algorithm, such as strong dependence on the initial population, premature convergence, and insufficient global search capability, this article develops a multi-strategy enhanced Parrot Optimization algorithm (MEPO). Experiments conducted on eight benchmark test functions validate the superior performance of MEPO in terms of convergence speed and solution accuracy. Secondly, to mitigate the adverse impact of irrelevant features on model performance in traditional software defect prediction methods, this study introduces a binary multi-strategy enhanced Parrot Optimization algorithm (BMEPO) for optimizing feature selection. Comparative experiments demonstrate that BMEPO exhibits stronger competitiveness in feature selection quality and classification performance compared to advanced feature selection algorithms. Finally, to further enhance the classification performance of defect prediction, a heterogeneous data stacking ensemble learning algorithm (HEDSE) based on feature selection is proposed. Experimental evaluations on 16 open-source software defect datasets indicate that the proposed HEDSE outperforms existing methods, providing a novel and effective solution for software defect prediction. The proposed approaches in this study hold significant practical value, particularly in improving software quality, optimizing testing resource allocation, and reducing maintenance costs, offering broad potential for application in real-world software engineering scenarios.

The development of a user-oriented procedure for software reliability prediction of telecommunications systems is proposed. The procedure is based on software reliability data that encapsulate the characteristics of the software decomposed into comparable units, as well as the more traditional failure data used in most of the current models. The state-of-the-art in software reliability prediction is reviewed. The software characteristics and the software reliability models that could form the backbone of a procedure are described together with some of the software reliability issues that have to be resolved for a useful prediction procedure for telecommunications.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Predicting Reliability of Software using SRGMs are a big challenge for Software Engineers. Software Reliability is a way to find the probability of software to operate in a given time limit and in a specified environment without causing any failure; hence it is reviewed as Quantifiable Metric. System faults can be the result of software or hardware errors. Software fault is comparatively more difficult to predict than hardware faults. Even the Reliability of a web application is also hard to determine due to its highly distributed nature. Predicting the reliability of application helps the Engineers to compute the software's release date/time and to manage various resources of software such as people, money, time etc. In this paper SRGM Models are considered namely Goel Okumoto Model (GO Model), Yamada S-shaped Model and Kapur & Garg Model (KG Model) to estimate and predict Software Reliability by detecting the cumulative number of faults in software application within specified time. Additional number of days are also calculated to remove them and release the software on time. By implementing the test cases of actual defects per day, Rate of Change or Test Case Efficiency can be measured.