Software Reliability Growth Models with Testing-Effort

Software reliability assessment for an exponential-S-shaped reliability growth phenomenon

Performance analysis of software reliability growth models with testing-effort and change-point

This investigation deals with a N-modules software reliability growth model (SRGM) wherein each module attains multiple types of faults of different severity levels. During the debugging phase, there may arise some new faults due to assumption of imperfect debugging. The reliability growth during the software testing phase governed by a non-homogeneous Poisson Process (NHPP). The proposed study is devoted to analyze the fault debugging phenomenon in all modules of software and to obtain expected number of faults observed, isolated and removed. The software reliability and optimum total cost of the software development during the testing phase are evaluated. The causes of fault occurrences in each module are different. The faults may occur with different failure rates and follow the nonlinear behavior with respect to the testing time. The NHPP model incorporates the testing-effort expenditure required for the faults debugging activities and governed by Weibull and logistic distributions which influence considerably not only the fault observation rate but also the time to isolate the faults and the time to remove the faults from all N-modules. In addition to predict the reliability of N-modular software, this study proposes optimal release policies for the SRGM by minimizing the expected cost subject to budget and testing-effort constraints, while ensuring a prescribed level of reliability and incorporating the effects of testing effort and efficiency. Some specific models are also deduced by setting the appropriate parameters so as to match with the existing models in the literature. Sensitivity results are presented through a numerical experiment to demonstrate the validity of the analytical formulations and the computational tractability of the model. An evolutionary algorithm, namely Particle Swarm Optimization (PSO) based on a dynamic penalty method, is employed to determine the optimal release time.

In order to overcome the limitations of debugging process, insufficient consideration of imperfect debugging and testing-effort (TE) in software reliability modeling and analysis, a software reliability growth model (SRGM) explicitly incorporating imperfect debugging and TE is developed. From the point of view of incomplete debugging and introduction of new fault, software testing process is described and a relatively unified SRGM framework is presented considering TE. The proposed framework models are fairly general models that cover a variety of the previous works on SRGM with ID and TE. Furthermore, a special SRGM incorporating an improved Logistic testing-effort function (TEF) into imperfect debugging modeling is proposed. The effectiveness and reasonableness of the proposed model are verified by published failure data set. The proposed model closer to real software testing has better descriptive and predictive power than other models

The software testing process basically aims at building confidence in the software for its use in real world applications. The reliability of a software system is always important to us. As we carry out the error detection and correction phenomenon on our software, the reliability of the software grows. With an aim to model this growth in the software reliability, many formulations in the form of Software Reliability Growth Models have been proposed. Many of these are based on Non-Homogeneous Poisson Process framework. In this paper, a parallel comparison of the performance of the proposed software reliability growth models is carried out, considering linear and exponential fault content functions for study of imperfect debugging situations. The performance of the proposed models has been compared with some famous existing software reliability models and the proposed models have been validated on some real-world datasets. Three goodness-of-fit criteria that include mean square error, predictive-ratio risk and predictive power are used to carry out the performance comparison of the models. Using these comparison criteria on six actual failure datasets, it is concluded that the proposed Model-2 which always outperforms Model-1, fits the actual failure data better and has better predictive power than other considered SRGMs for at least two data sets.

The safety-critical software of Reactor Protection System (RPS) plays a significant role for the safe operation of the nuclear power plant (NPP). However, it also brings challenges both to the reliability analysis of the RPS and to the Probabilistic Risk Assessment of the NPPs. The reliability analysis of safety-critical software is also expected by the nuclear regulation agencies and the software development groups for test evaluation and optimization. The detected faults during the software test process are regarded to have close connection with the software reliability and there have been hundreds of test-based software reliability models. Due to the particularity of its function, the safety-critical software of an NPP is especially sensitive to the faults with higher severity levels which should be paid special attention to. Severity levels of faults are commonly taken into account during software reliability modelling. In this paper, a novel software reliability growth model considering actual severity data was built based on a non-homogeneous Poisson process. Ratio of fatal faults with highest severity level to all accumulated faults was modelled with logistic curve. Then the mean value functions of both fatal and general faults were derived. A belief factor was employed to describe the trend of severity classification. In the end, the test data of a practical project was used to verify this model. The analysis results showed that this model had better prediction effect than Goel-Okumoto model and Inflection S-shaped model even when limited test data were collected. This novel model can be used as a helpful tool to evaluate both the reliability and the release time of the safety-critical software of an NPP.

A large number of software reliability growth models (SRGMs) have been proposed during the past 30 years to estimate software reliability measures such as the number of remaining faults, software failure rate, and software reliability. Selection of an optimal SRGM for use in a particular case has been an area of interest for researchers in the field of software reliability. Tools and techniques for software reliability model selection found in the literature cannot be used with high confidence as they use a limited number of model selection criteria. For the first time, we developed a deterministic quantitative model based on a distance based approach (DBA) method, then applied it for evaluation, optimal selection, and ranking of SRGMs. DBA recognizes the need for relative importance of criteria for a given application, without which inter-criterion comparison could not be accomplished. It requires a set of model selection criteria, along with a set of SRGMs, and their level of criteria for optimal selection; and it successfully presents the results in terms of a merit value which is used to rank the SRGMs. We use two distinct, real data sets for demonstration of the DBA method. The result of this study will be a ranking of SRGMs based on the Euclidean composite distance of each alternative to the designated optimal SRGM.

A software reliability growth model incorporating the amount of test effort expenditures during the software testing phase is described. The time-dependent behavior of test-effort expenditures is described by a Weibull curve due to the flexibility. Assuming that the error detection rate to the amount of test-effort spent during the testing phase is proportional to the current error content, the model is formulated by a nonhomogeneous Poisson process. Using this model, the method of data analysis for software reliability measurement is presented. Also, this model is applied to the prediction of additional test-effort expenditures to obtain the objective number of errors detected by software testing and the determination of the optimum time to stop software testing for release. >

It is one of the important issues in modern software development to make a highly reliable software system by efficient and economical testing. As one of these solutions, many companies have taken in testing-progress management and quality/reliability assessment based on software reliability growth models. Consequently, many software reliability growth models reflecting various development factors and operational environment of the software system have been proposed. Most of these models have assumed the software system as one domain. That is, the models have assumed that the set of the testing-paths influenced by execution of test-cases is extended with testing-progress and it expands to whole software system finally. However, a software system has the structure that the main-component calls plural sub-components ordinarily. Therefore, unless the main-component is tested, the sub-component called from it is not tested. In this chapter, we propose software reliability growth models based on such components' characteristics. Especially, these models reflecting the different testing-environment for composed components are formulated by nonhomogeneous Poisson processes. Furthermore, using fault-detection data observed in actual development projects, we show numerical examples of software reliability assessment and results of goodness-of-fit comparisons with existing software reliability growth models.

Multiple Software Reliability Growth Models (SRGM) has been illustrated to inspire designers and software developers to develop quality software. These models implicitly assume that expenditure is constant throughout testing effort. Recently importance was given to the testing phase by the development team and it requires controlled screening closely to raise its overall operational efficiency. Software Reliability Growth Models exhibits a precise relation between the failure patterns with time. Considering this as a basic building block a SRGM which is a Non-Homogeneous Poisson Process in nature is i in this work. The various aspects of our proposed model are testing effort function and fault detection rate (FDR). Mainly in Software Reliability Growth Model it was assumed that fault removal process are perfect which is not true in reality as new faults may get introduced during removal or some faults may not get removed due to its complexity. Thus in this model the concept of imperfect debugging was considered and assists us to justify the software reliability more convincingly through the help of testing effort function.

Software reliability assessment models based on cumulative bernoulli trial processes

In order to incorporate the effect of test coverage, two novel software reliability growth models (SRGMs) are proposed in this paper using failure data and test coverage simultaneously. One is continuous using testing time, and the other is discrete with respect to the number of executed test cases instead of testing time. Since one of the most important factors of the coverage-based SRGMs is the test coverage function (TCF), we first discuss a discrete TCF based on Beta function. Then we develop mean value functions (MVF) of the two models integrating test coverage and imperfect debugging. Finally the proposed TCF and MVFs are evaluated and validated on actual software reliability data collected from real software development projects. The results demonstrate clearly that both the proposed TCF and SRGMs provide better estimation and fitting for the data sets under comparisons.

In reality, the fault detection (or correction) phenomenon and software reliability estimation in the operational phase are different from that in the testing phase. The fault removal continues at a slower rate during the operational phase. In this paper, we will investigate some techniques for software reliability prediction and measurement in the operational phase. We first review how some software reliability growth models (SRGMs) based on non-homogeneous Poisson processes (NHPPs) can be readily derived based on a unified theory for NHPP models. Under this general framework, we can not only verify some conventional SRGMs but also derive some new SRGMs that can be used for software reliability measurement in the operational phase. That is, based on the unified theory, we can incorporate the concept of multiple changepoints into software reliability modeling. We can formularize and simulate the fault detection process in the operational phase. Some numerical illustrations based on real software failure data are also presented. The experimental results show that the proposed models can easily reflect the possible changes of fault detection process and offer quantitative analysis on software failure behavior in field operation.

With the growth of complexity in the software system, to deliver reliable software products on time becomes a critical issue. Many software reliability growth models (SRGMs) have been proposed in the past three decades. However, most software reliability assessment processes and parameter estimations of models depend on the computations of the general-purposed numerical software. In this paper, we will present a powerful computer-aided reliability assessment tool for software based on object-oriented analysis and design-CARATS. CARATS can use both traditional SRGMs and neural-network methods to assess software reliability. This would greatly help project managers to make decisions during software development life cycle. Due to the characteristics of the special-purposed and object-oriented design, CARATS can analyze the software reliability easily, and is also more flexible to adopt different SRGMs than other existing software reliability analysis tools.

Measuring reliability refers to quantifying the reliability of a component or system by quantitative metrics. Software reliability growth models have been proposed to estimate and predict the reliability growth of software. Most conventional Software reliability growth models take the faults detected may be corrected immediately into consideration. In reality, this case may not be reasonable and may not be always occur, as there is often a considerable time delay between identify the root causes of faults detected and then correct them. Besides, experiments show that detection effort, correction effort, and the match of detection effort and correction effort have the great effect on time delay. In this paper, we use finite server queuing models to investigate software debugging processes. We will propose finite server queuing models with various distribution function and change-point under perfect and imperfect debugging environments. An actual software failure data set is analyzed to illustrate the effectiveness of proposed models. Experimental results show that the proposed models have a fairly accurate predication capability.