Release Planning Analysis Through Testing Coverage and Fault Reduction Factor Based Models with Change Point Perspective

In reality, the fault detection (or correction) phenomenon and software reliability estimation in the operational phase are different from that in the testing phase. The fault removal continues at a slower rate during the operational phase. In this paper, we will investigate some techniques for software reliability prediction and measurement in the operational phase. We first review how some software reliability growth models (SRGMs) based on non-homogeneous Poisson processes (NHPPs) can be readily derived based on a unified theory for NHPP models. Under this general framework, we can not only verify some conventional SRGMs but also derive some new SRGMs that can be used for software reliability measurement in the operational phase. That is, based on the unified theory, we can incorporate the concept of multiple changepoints into software reliability modeling. We can formularize and simulate the fault detection process in the operational phase. Some numerical illustrations based on real software failure data are also presented. The experimental results show that the proposed models can easily reflect the possible changes of fault detection process and offer quantitative analysis on software failure behavior in field operation.

Software testing involves verification and validation of the software to meet the requirements elucidated by customers in the earlier phases and to subsequently increase software reliability. Around half of the resources, such as manpower and CPU time are consumed and a major portion of the total cost of developing the software is incurred in testing phase, making it the most crucial and time-consuming phase of a software development lifecycle (SDLC). Also the fault detection process (FDP) and fault correction process (FCP) are the important processes in SDLC. A number of software reliability growth models (SRGM) have been proposed in the last four decades to capture the time lag between detected and corrected faults. But most of the models are discussed under static environment. The purpose of this paper is to allocate the resources in an optimal manner to minimize the cost during testing phase using FDP and FCP under dynamic environment. An elaborate optimization policy based on optimal control theory for resource allocation with the objective to minimize the cost is proposed. Further, genetic algorithm is applied to obtain the optimum value of detection and correction efforts which minimizes the cost. Numerical example is given in support of the above theoretical result. The experimental results help the project manager to identify the contribution of model parameters and their weight.

Owing to release of software in multiple releases, code changes take place in software. Because of this added complexity in software, the testing team may be unable to correct the fault upon detection, leaving the actual fault to reside in the software, termed as imperfect debugging or there may be replacement of original fault by other fault, leading to error generation. Many other factors exist that affect the testing phase of software like strategies of testing, test cases, skill, efficiency, and learning of testing team. All these factors cannot be kept stable during the whole process of testing. They may change at any time moment causing the background processes to experience change, which is known as change‐point. Keeping all these critical testing environment factors under consideration, a new software reliability growth model has been proposed, which is derived from an non homogenous Poisson process (NHPP)based unified scheme for multi‐release two‐stage fault detection/observation and correction/removal software reliability models. The developed model is numerically illustrated on tandem data set for four releases.

Reliability is the one of the important quality measure of software product. Testing is one important phase in software development life cycle whose intention is to find the bugs present in the software product. A mathematical model which describes the software testing in software development cycle termed as software reliability growth models. Past few decades many software reliability growth models were proposed. Estimating the accurate release time of software is an important challenging issue of software Development Company. If the software product is released early contain more errors makes software to be less reliable, where as late release will increase the development cost. It is observed that more than half of the total development cost is concentrated during the maintenance phase. Estimating the reliability and accurate development cost of software product by software reliability growth model at maintenance phase is an important issue. Warranty is defined to be an agreement between the customer and vendor of the software to provide the extra protection to the product. Several papers have concentrated on optimal release time by considering the warranty cost. All early proposed NHPP warranty cost models considered a perfect-debugging software reliability growth model while estimating the software maintenance cost. But it was observed that, the software product is influenced by several factors like environment, resources and nature of the faults during the operational phase. In such circumstances it is quite not right to consider a perfect NHPP software reliability growth model in operational phase. In this paper we proposed an imperfect debugging software reliability growth model by combining the cost and reliability at a given warranty period to estimate the optimal release time of software product.

Estimating remaining defects (or failures) in software can help test managers make release decisions during testing. Several methods exist to estimate defect content, among them a variety of software reliability growth models (SRGMs). SRGMs have underlying assumptions that are often violated in practice, but empirical evidence has shown that many are quite robust despite these assumption violations. The problem is that, because of assumption violations, it is often difficult to know which models to apply in practice. We present an empirical method for selecting SRGMs to make release decisions. The method provides guidelines on how to select among the SRGMs to decide on the best model to use as failures are reported during the test phase. The method applies various SRGMs iteratively during system test. They are fitted to weekly cumulative failure data and used to estimate the expected remaining number of failures in software after release. If the SRGMs pass proposed criteria, they may then be used to make release decisions. The method is applied in a case study using defect reports from system testing of three releases of a large medical record system to determine how well it predicts the expected total number of failures.

In order to incorporate the effect of test coverage, two novel software reliability growth models (SRGMs) are proposed in this paper using failure data and test coverage simultaneously. One is continuous using testing time, and the other is discrete with respect to the number of executed test cases instead of testing time. Since one of the most important factors of the coverage-based SRGMs is the test coverage function (TCF), we first discuss a discrete TCF based on Beta function. Then we develop mean value functions (MVF) of the two models integrating test coverage and imperfect debugging. Finally the proposed TCF and MVFs are evaluated and validated on actual software reliability data collected from real software development projects. The results demonstrate clearly that both the proposed TCF and SRGMs provide better estimation and fitting for the data sets under comparisons.

A software release problem based on four software reliability growth models (SRGMs) with random life-cycle length is studied. Test of the software system is terminated after time T and released (sold) to the user at a price. The price of the software system and three cost components are considered, and average total profit is used as a criterion. The optimal values of release times are shown to be finite and unique. Hence, the optimal solutions can be obtained numerically by, for example, a bisection method. A numerical example indicates that the optimal release time increases as (1) the error rate in each model decreases and (2) the difference between the error fixing cost during the test phase and that during the operational phase increases. The case of unknown model parameters is considered only for the Jelinski-Moranda model because a Bayes model is not available for other SRGMs. The release decision depends on testing time, but other stopping rules, for example based on the number of corrected errors, can be considered.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

PurposeThe aim of this paper is to propose a software reliability growth model based on stochastic differential equations for the integration testing phase of distributed development environment.Design/methodology/approachA client/server system (CSS), which is a new development method, has come into existence as a result of the progress of networking technology by UNIX systems. On the other hand, the effective testing method for distributed development environment has only a few presented. The method of software reliability assessment considering the interaction among software components in a distributed one is discussed.FindingsConventional software reliability growth models for system testing phase in distributed development environment have included many unknown parameters. Especially, the effective estimation method in terms of these unknown parameters, which means the proportion of the total testing‐load for the software component, has never been presented. This software reliability growth model can be easily applied in distributed software development, because the model has a simple structure.Practical implicationsThis model is very useful for software developers in terms of practical reliability assessment in the actual distributed development environment.Originality/valueThe method of software reliability assessment considering the interaction among software components in distributed development environment is proposed. Additionally, several numerical examples for the actual data are presented.

This investigation deals with a N-modules software reliability growth model (SRGM) wherein each module attains multiple types of faults of different severity levels. During the debugging phase, there may arise some new faults due to assumption of imperfect debugging. The reliability growth during the software testing phase governed by a non-homogeneous Poisson Process (NHPP). The proposed study is devoted to analyze the fault debugging phenomenon in all modules of software and to obtain expected number of faults observed, isolated and removed. The software reliability and optimum total cost of the software development during the testing phase are evaluated. The causes of fault occurrences in each module are different. The faults may occur with different failure rates and follow the nonlinear behavior with respect to the testing time. The NHPP model incorporates the testing-effort expenditure required for the faults debugging activities and governed by Weibull and logistic distributions which influence considerably not only the fault observation rate but also the time to isolate the faults and the time to remove the faults from all N-modules. In addition to predict the reliability of N-modular software, this study proposes optimal release policies for the SRGM by minimizing the expected cost subject to budget and testing-effort constraints, while ensuring a prescribed level of reliability and incorporating the effects of testing effort and efficiency. Some specific models are also deduced by setting the appropriate parameters so as to match with the existing models in the literature. Sensitivity results are presented through a numerical experiment to demonstrate the validity of the analytical formulations and the computational tractability of the model. An evolutionary algorithm, namely Particle Swarm Optimization (PSO) based on a dynamic penalty method, is employed to determine the optimal release time.

With the growing competition and the demand of the customers, a software organization needs to regularly provide up-gradations and add features to its existing version of software. For the organization, creating these software upgrades means an increase in the complexity of the software which in turn leads to the increase in the number of faults. Also, the faults left undetected in the previous version need to be addressed in this phase. Many software reliability growth models have been proposed to model the phenomenon of multi-release problems using two stage failure observation and correction processes. The model proposed in this paper partitions the fault removal process into a two-stage process which includes fault detection process and fault removal process considering the joint effect of premeditated release pressure and resource restrictions using a well-known Cobb–Douglas production function for the multi release problem of a software. The faults detected in the operational phase of the previous release or left incomplete are also incorporated in the next release. A generalized framework for the multi-release problem in which fault detection follows an exponential distribution function and fault correction follows Gamma distribution function is proposed and verified on a real data set of four releases of software. The estimated parameters and comparison criteria are also given.

In this chapter we discuss the software reliability growth models (SRGMs) that describe the relationship between the number of faults removed and the number of test cases used. Firstly we describe the discrete exponential and S-shaped models in a perfect debugging environment. We also discuss flexible discrete SRGM, which can depict either exponential or S-shaped growth curves, depending upon the parameter values estimated from the past failure data. Further, we describe an SRGM for the fault removal phenomenon in a perfect debugging environment. Most testing processes are imperfect in practice, therefore we also discuss a discrete model that incorporates the impact of imperfect debugging and fault generation into software reliability growth modeling. Faults in the software are generally not of the same type, rather the faults contained in a large software may differ from each other in terms of the amount of time and skill of the removal team required to remove them. Three discrete models: the generalized Erlang model, modeling severity of faults with respect to testing time, and a model with faults of different severity incorporating logistic learning function have been discussed. A discrete model in a distributed environment is also discussed. The above discrete SRGMs assume a constant fault detection rate while testing the software under consideration. In practice, however, the fault detection rate varies because of changes in the testing skill, the system environment, and the testing strategy used to test the software. SRGMs for the fault removal phenomenon, the generalized Erlang model, and the generalized Erlang model with logistic function are discussed, incorporating the concept of change point. It is also shown how equivalent continuous models can be derived. This chapter describes the state-of-the-art in discrete modeling.

Over the past 30 years, many software reliability growth models (SRGMs) have been proposed for estimation of reliability growth of software. In fact, effective debugging will not be easy because the fault may not be immediately obvious. In the past, some researchers ever used an infinite server queueing (ISQ) model to describe the software debugging behavior. An infinite-server queueing model is considered where access of customers to service is controlled by a gate and the gate is open only if all servers are free. However, the finite server queueing (FSQ) model is first advantageously modeled as an infinite-server system. Thus, in this paper, we will show how to incorporate both FSQ and ISQ models into software reliability estimation and prediction. In addition, here we also consider the factor of perfect/imperfect debugging. Experimental results show that the proposed framework to incorporate both fault detection and correction processes for SRGM has a fairly accurate prediction capability.

In today's fast moving day to day life almost everything is more or less dependent on software. This dependence on software has increased the demand of reliable software in no time. Hence software firms are now a days releasing multiple versions of the software instead of providing a software with less number of bugs and removing the bugs of the previous version in the next up-gradation Firms remove the bugs of the previous version in the next up-gradation or by updating the software. But removal of bugs is not a simple task and at the time of removal it may be possible that testing team may introduce some new bugs to it. In the existing software reliability literature numerous models are proposed incorporating effort spent on removing the bugs. In this paper we have incorporated one more attribute of fault reduction factor in the effort based multi release modelling of a software. Fault reduction factors (FRFs) play an important role in software reliability and are generally defined as the ratio of the total number of reduced faults to the total number of failures experienced. FRFs are not fixed and it is affected by many factors, e.g., imperfect debugging, resource allocations, and debugging time lag. Taking this into consideration we propose two effort based multi up-gradation software reliability growth models incorporating time dependent FRF. Numerical illustration of the proposed model is done on a real life data set of tandem computers for four release.

This paper discusses software reliability growth modeling with effect of changing the software failure-occurrence phenomenon in the testing phase, and checks the software reliability measurement accuracy of our models by using actual data. Especially, our modeling approach in this paper uses a testing-environmental function for describing a relationship between the software failure-occurrence time intervals before the change and those after the change. Further, we discuss a problem for estimating the optimal changing-time and software shipping-time simultaneously from the point of view of the total expected software cost expending in the testing and operational phases. Finally, we show examples of the application of software reliability measurement based on our model and those of the derived optimal policy by using actual data.

As modern software system is growing in size and complexity, the customer expectations for software quality have become higher. In the past, many software reliability growth models (SRGMs) were proposed and they are helped to evaluate the quality of developed software. It is worth noting that some of SRGMs can be used to model the fault detection process (FDP) and the fault correction process (FCP) through an infinite server queueing (ISQ) system or a finite server queueing (FSQ) system. However, it can also be found that most ISQ and FSQ models were developed on a first come first served basis. In this paper, we propose to use the queueing-based simulations to describe the behavior of FCP and assess the software reliability instead of using model-based approaches. Our proposed queueing-based simulation techniques and simulation procedures will be able to thoroughly investigate the FCP and easily provide system performance information estimated based on the staffing level, the average response time, and the average waiting time. Numerical examples based on three real failure data are given and discussed. Our experiments show that the proposed simulation procedures obtain a good prediction capability for software reliability. We expect that the proposed methods can provide effective information for software developing management and help decision makers in resource allocation and cost control.