An Empirical Method for Selecting Software Reliability Growth Models

Researchers have proposed and implemented a plethora of bug prediction approaches in terms of different mathematical models for measuring the reliability growth of the software and to predict the latent bugs lying dormant in the software. During the last four decades, software reliability growth models (SRGM) have been successfully used to measure the reliability growth of closed source software. The SRGM developed were based on either calendar time or on testing effort. In late 90s, due to the advancement in communication and internet technologies, the development of open source software gets an edge and is proven to be very successful in different fields. Recently, researchers have measured the latent bugs in the open source software using an SRGM which has been developed for closed source software and concluded that the existing SRGM can well predict the latent bugs, but, still, it needs more investigation. In open source software, the source codes are frequently changes (the complexity of code changes) to meet the new feature introduction, feature enhancement and bug repair. In this paper, we have developed two complexity of code changes/entropy based bug prediction models namely (i) time vs entropy and (ii) entropy vs bugs. We have compared the proposed models with the existing time vs bugs SRGM. The empirical work has been carried out using three subsystems of Mozilla project. The statistical significance of different approaches has been tested using a non-parametric Kolmogorov–Smirnov (K–S) test. The bug prediction approaches have been compared on the basis of various performance measures namely R-Square (R2), Adjusted R-Square (adj. R2), Bias, variation and root mean square prediction errors. We found that the potential complexity of code changes based bug prediction approach i.e. time vs entropy is better over the time vs bugs and entropy vs bugs on the basis of different comparison criteria and statistical test.

This paper proposes software reliability growth models (SRGM) where the software failure time follows a normal distribution. The proposed model is mathematically tractable and has sufficient ability of fitting to the software failure data. In particular, we consider the parameter estimation algorithm for the SRGM with normal distribution. The developed algorithm is based on an EM (expectation-maximization) algorithm and is quite simple for implementation as software application. Numerical examples are devoted to investigating the fitting ability of normal distribution based SRGMs through software failure data collected in real software projects.

In reality, the fault detection (or correction) phenomenon and software reliability estimation in the operational phase are different from that in the testing phase. The fault removal continues at a slower rate during the operational phase. In this paper, we will investigate some techniques for software reliability prediction and measurement in the operational phase. We first review how some software reliability growth models (SRGMs) based on non-homogeneous Poisson processes (NHPPs) can be readily derived based on a unified theory for NHPP models. Under this general framework, we can not only verify some conventional SRGMs but also derive some new SRGMs that can be used for software reliability measurement in the operational phase. That is, based on the unified theory, we can incorporate the concept of multiple changepoints into software reliability modeling. We can formularize and simulate the fault detection process in the operational phase. Some numerical illustrations based on real software failure data are also presented. The experimental results show that the proposed models can easily reflect the possible changes of fault detection process and offer quantitative analysis on software failure behavior in field operation.

In the software industry, Software Reliability Growth Models (SRGMs) with confidence intervals (C.I.) are frequently employed as valuable tools to assist manager to determine the optimal timing for software releases and enhancing the effectiveness of testing processes. However, many SRGMs lack clear explanations regarding the estimation of variance for cumulative software errors. This limitation can impede their ability to accurately derive C.I. for the mean value function (MVF), making it challenging for developers to evaluate potential risks and uncertainties within the software system. Consequently, the practical application of these models may be compromised. To address this issue, this paper proposes the utilization of Ohba’s Inflection S-shaped model and Yamada’s delayed S-shaped model to construct SRGMs with C.I., thereby providing developers with improved guidance for making optimal release time decisions.

A large number of software reliability growth models (SRGMs) have been proposed during the past 30 years to estimate software reliability measures such as the number of remaining faults, software failure rate, and software reliability. Selection of an optimal SRGM for use in a particular case has been an area of interest for researchers in the field of software reliability. Tools and techniques for software reliability model selection found in the literature cannot be used with high confidence as they use a limited number of model selection criteria. For the first time, we developed a deterministic quantitative model based on a distance based approach (DBA) method, then applied it for evaluation, optimal selection, and ranking of SRGMs. DBA recognizes the need for relative importance of criteria for a given application, without which inter-criterion comparison could not be accomplished. It requires a set of model selection criteria, along with a set of SRGMs, and their level of criteria for optimal selection; and it successfully presents the results in terms of a merit value which is used to rank the SRGMs. We use two distinct, real data sets for demonstration of the DBA method. The result of this study will be a ranking of SRGMs based on the Euclidean composite distance of each alternative to the designated optimal SRGM.

Many software reliability growth models have been proposed in the past decade. Those models tacitly assume that testing-effort expenditures are constant throughout software testing. This paper develops realistic software reliability growth models incorporating the effect of testing-effort. The software error detection phenomenon in software testing is modeled by a nonhomogeneous Poisson process. The software reliability assessment measures and the estimation methods of parameters are investigated. Testing-effort expenditures are described by exponential and Rayleigh curves. Least-squares estimators and maximum likelihood estimators are used for the reliability growth parameters. The software reliability data analyses use actual data. The software reliability growth models with testing-effort can consider the relationship between the software reliability growth and the effect of testing-effort. Thus, the proposed models will enable us to evaluate software reliability more realistically.

Reliability is the one of the important quality measure of software product. Testing is one important phase in software development life cycle whose intention is to find the bugs present in the software product. A mathematical model which describes the software testing in software development cycle termed as software reliability growth models. Past few decades many software reliability growth models were proposed. Estimating the accurate release time of software is an important challenging issue of software Development Company. If the software product is released early contain more errors makes software to be less reliable, where as late release will increase the development cost. It is observed that more than half of the total development cost is concentrated during the maintenance phase. Estimating the reliability and accurate development cost of software product by software reliability growth model at maintenance phase is an important issue. Warranty is defined to be an agreement between the customer and vendor of the software to provide the extra protection to the product. Several papers have concentrated on optimal release time by considering the warranty cost. All early proposed NHPP warranty cost models considered a perfect-debugging software reliability growth model while estimating the software maintenance cost. But it was observed that, the software product is influenced by several factors like environment, resources and nature of the faults during the operational phase. In such circumstances it is quite not right to consider a perfect NHPP software reliability growth model in operational phase. In this paper we proposed an imperfect debugging software reliability growth model by combining the cost and reliability at a given warranty period to estimate the optimal release time of software product.

Number of software Reliability growth models has been proposed in the literature. A mathematical technique which describes the software testing phenomenon known as the software reliability growth model. Software reliability growth models are used to predict the number of faults and reliability of the software. In the view of this software reliability growth models are basically differentiated as the continuous and discrete models. There is a plenty of development in the continuous models but little towards the discrete models. In this paper we have presented a discrete reliability growth model with different discrete testing effort functions and the same time software release policy is discussed. A new imperfect debugging discrete software reliability growth model with testing effort is proposed. All calculations are done on real data. The results shows the proposed testing effort models are perfectly fit to the data.

In this chapter we discuss the software reliability growth models (SRGMs) that describe the relationship between the number of faults removed and the number of test cases used. Firstly we describe the discrete exponential and S-shaped models in a perfect debugging environment. We also discuss flexible discrete SRGM, which can depict either exponential or S-shaped growth curves, depending upon the parameter values estimated from the past failure data. Further, we describe an SRGM for the fault removal phenomenon in a perfect debugging environment. Most testing processes are imperfect in practice, therefore we also discuss a discrete model that incorporates the impact of imperfect debugging and fault generation into software reliability growth modeling. Faults in the software are generally not of the same type, rather the faults contained in a large software may differ from each other in terms of the amount of time and skill of the removal team required to remove them. Three discrete models: the generalized Erlang model, modeling severity of faults with respect to testing time, and a model with faults of different severity incorporating logistic learning function have been discussed. A discrete model in a distributed environment is also discussed. The above discrete SRGMs assume a constant fault detection rate while testing the software under consideration. In practice, however, the fault detection rate varies because of changes in the testing skill, the system environment, and the testing strategy used to test the software. SRGMs for the fault removal phenomenon, the generalized Erlang model, and the generalized Erlang model with logistic function are discussed, incorporating the concept of change point. It is also shown how equivalent continuous models can be derived. This chapter describes the state-of-the-art in discrete modeling.

PurposeThe aim of this paper is to propose a software reliability growth model based on stochastic differential equations for the integration testing phase of distributed development environment.Design/methodology/approachA client/server system (CSS), which is a new development method, has come into existence as a result of the progress of networking technology by UNIX systems. On the other hand, the effective testing method for distributed development environment has only a few presented. The method of software reliability assessment considering the interaction among software components in a distributed one is discussed.FindingsConventional software reliability growth models for system testing phase in distributed development environment have included many unknown parameters. Especially, the effective estimation method in terms of these unknown parameters, which means the proportion of the total testing‐load for the software component, has never been presented. This software reliability growth model can be easily applied in distributed software development, because the model has a simple structure.Practical implicationsThis model is very useful for software developers in terms of practical reliability assessment in the actual distributed development environment.Originality/valueThe method of software reliability assessment considering the interaction among software components in distributed development environment is proposed. Additionally, several numerical examples for the actual data are presented.

This investigation deals with a N-modules software reliability growth model (SRGM) wherein each module attains multiple types of faults of different severity levels. During the debugging phase, there may arise some new faults due to assumption of imperfect debugging. The reliability growth during the software testing phase governed by a non-homogeneous Poisson Process (NHPP). The proposed study is devoted to analyze the fault debugging phenomenon in all modules of software and to obtain expected number of faults observed, isolated and removed. The software reliability and optimum total cost of the software development during the testing phase are evaluated. The causes of fault occurrences in each module are different. The faults may occur with different failure rates and follow the nonlinear behavior with respect to the testing time. The NHPP model incorporates the testing-effort expenditure required for the faults debugging activities and governed by Weibull and logistic distributions which influence considerably not only the fault observation rate but also the time to isolate the faults and the time to remove the faults from all N-modules. In addition to predict the reliability of N-modular software, this study proposes optimal release policies for the SRGM by minimizing the expected cost subject to budget and testing-effort constraints, while ensuring a prescribed level of reliability and incorporating the effects of testing effort and efficiency. Some specific models are also deduced by setting the appropriate parameters so as to match with the existing models in the literature. Sensitivity results are presented through a numerical experiment to demonstrate the validity of the analytical formulations and the computational tractability of the model. An evolutionary algorithm, namely Particle Swarm Optimization (PSO) based on a dynamic penalty method, is employed to determine the optimal release time.

Software reliability assessment models based on cumulative bernoulli trial processes

A software release problem based on four software reliability growth models (SRGMs) with random life-cycle length is studied. Test of the software system is terminated after time T and released (sold) to the user at a price. The price of the software system and three cost components are considered, and average total profit is used as a criterion. The optimal values of release times are shown to be finite and unique. Hence, the optimal solutions can be obtained numerically by, for example, a bisection method. A numerical example indicates that the optimal release time increases as (1) the error rate in each model decreases and (2) the difference between the error fixing cost during the test phase and that during the operational phase increases. The case of unknown model parameters is considered only for the Jelinski-Moranda model because a Bayes model is not available for other SRGMs. The release decision depends on testing time, but other stopping rules, for example based on the number of corrected errors, can be considered.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

In order to overcome the limitations of debugging process, insufficient consideration of imperfect debugging and testing-effort (TE) in software reliability modeling and analysis, a software reliability growth model (SRGM) explicitly incorporating imperfect debugging and TE is developed. From the point of view of incomplete debugging and introduction of new fault, software testing process is described and a relatively unified SRGM framework is presented considering TE. The proposed framework models are fairly general models that cover a variety of the previous works on SRGM with ID and TE. Furthermore, a special SRGM incorporating an improved Logistic testing-effort function (TEF) into imperfect debugging modeling is proposed. The effectiveness and reasonableness of the proposed model are verified by published failure data set. The proposed model closer to real software testing has better descriptive and predictive power than other models

One of the established trends of research areas and practices in software engineering that dealt with the measurement and enhancement of reliability is software reliability engineering. Stochastic software reliability models find typical usage for analysis. These are the models that perform modeling on failure process of the software and exploit other software metrics or failure data as cornerstone for parameter estimation. The ability of the models for estimating and predicting the current reliability and future failure behavior, respectively, is high. Due to any failures and faults in the system, a product becomes unreliable. The lack of understanding of nature of the software makes the measurement of software reliability as a challenging task. It is not possible to determine a best way to measure the reliability and other aspects of software. This paper proposes an efficient software reliability growth model (SRGM) in which logistic exponential TEF is exploited. This model offers increased failure rate recognition and suitable ways to resolve faults and so on. Our work estimates the SRGM parameters using optimization algorithm. Such estimation can aid in developing precise software reliability model. In order to accomplish the optimization, we use artificial bee colony (ABC) algorithm. As the parameters optimization considerably improves the quality of parameters to be used for reliability growth model, reliability growth can also be improved considerably.