Considering fault removal efficiency in software reliability assessment

In order to overcome the limitations of debugging process, insufficient consideration of imperfect debugging and testing-effort (TE) in software reliability modeling and analysis, a software reliability growth model (SRGM) explicitly incorporating imperfect debugging and TE is developed. From the point of view of incomplete debugging and introduction of new fault, software testing process is described and a relatively unified SRGM framework is presented considering TE. The proposed framework models are fairly general models that cover a variety of the previous works on SRGM with ID and TE. Furthermore, a special SRGM incorporating an improved Logistic testing-effort function (TEF) into imperfect debugging modeling is proposed. The effectiveness and reasonableness of the proposed model are verified by published failure data set. The proposed model closer to real software testing has better descriptive and predictive power than other models

A large number of software reliability growth models (SRGMs) have been proposed during the past 30 years to estimate software reliability measures such as the number of remaining faults, software failure rate, and software reliability. Selection of an optimal SRGM for use in a particular case has been an area of interest for researchers in the field of software reliability. Tools and techniques for software reliability model selection found in the literature cannot be used with high confidence as they use a limited number of model selection criteria. For the first time, we developed a deterministic quantitative model based on a distance based approach (DBA) method, then applied it for evaluation, optimal selection, and ranking of SRGMs. DBA recognizes the need for relative importance of criteria for a given application, without which inter-criterion comparison could not be accomplished. It requires a set of model selection criteria, along with a set of SRGMs, and their level of criteria for optimal selection; and it successfully presents the results in terms of a merit value which is used to rank the SRGMs. We use two distinct, real data sets for demonstration of the DBA method. The result of this study will be a ranking of SRGMs based on the Euclidean composite distance of each alternative to the designated optimal SRGM.

Software failure-occurrence or the fault-detection phenomenon is changed notably in an actual testing phase or an operational phase due to the changes of factors influencing the software reliability growth process. Taking into consideration of the effect of the change in software reliability growth modeling is expected to conduct more accurate software reliability assessment because it is said that such approach enables us to conduct more plausible software reliability assessment reflecting an actual testing-environment. We develop a framework for developing software hazard rate models with effect of change-point, and discuss change-point detection methods for applying our model to quantitative software reliability assessment. Additionally, we discuss an optimal software release problem for estimating optimal shipping time with the effect of the change as one of the application problems of our model in software project management. Finally, we show numerical examples for software reliability assessment and our optimal software release policy based on our change-point model by using actual data.

Comparing the reliability of software systems: A case study on mobile operating systems

Over the past three decades, many software reliability growth models (SRGMs) were proposed and they are aimed at predicting and estimating software reliability. One common assumption of these conventional SRGMs is that detected faults will be removed immediately. In reality, this assumption may not be reasonable and may not always occur. Developers need time to identify the root causes of detected faults and then fix them. Besides, during debugging the fault correction rate may not be a constant and could be changed at some certain points as time proceeds. Consequently, in this paper, we will explore and study how to apply queueing model to investigate the fault correction process during software development. We propose an extended infinite server queueing model with multiple change-points to predict and assess software reliability. Experimental results based on real failure data show that proposed model can depicts the change of fault correction rates and predict the behavior of software development more accurately than traditional SRGMs.

Software reliability assessment for an exponential-S-shaped reliability growth phenomenon

In order to incorporate the effect of test coverage, two novel software reliability growth models (SRGMs) are proposed in this paper using failure data and test coverage simultaneously. One is continuous using testing time, and the other is discrete with respect to the number of executed test cases instead of testing time. Since one of the most important factors of the coverage-based SRGMs is the test coverage function (TCF), we first discuss a discrete TCF based on Beta function. Then we develop mean value functions (MVF) of the two models integrating test coverage and imperfect debugging. Finally the proposed TCF and MVFs are evaluated and validated on actual software reliability data collected from real software development projects. The results demonstrate clearly that both the proposed TCF and SRGMs provide better estimation and fitting for the data sets under comparisons.

Over the past 30 years, many software reliability growth models (SRGMs) have been proposed for estimation of reliability growth of software. In fact, effective debugging will not be easy because the fault may not be immediately obvious. In the past, some researchers ever used an infinite server queueing (ISQ) model to describe the software debugging behavior. An infinite-server queueing model is considered where access of customers to service is controlled by a gate and the gate is open only if all servers are free. However, the finite server queueing (FSQ) model is first advantageously modeled as an infinite-server system. Thus, in this paper, we will show how to incorporate both FSQ and ISQ models into software reliability estimation and prediction. In addition, here we also consider the factor of perfect/imperfect debugging. Experimental results show that the proposed framework to incorporate both fault detection and correction processes for SRGM has a fairly accurate prediction capability.

This paper discusses two-dimensional software reliability measurement technologies, which describe a software reliability growth process depending on two-types of software reliability growth factors: Testing-time and testing-effort factors. From the view point of actual software failure-occurrence mechanism, it is natural to consider that a software reliability growth process depends on not only testing-time factor but also other software reliability growth factors, i.e., testing-effort factors, such as test-execution time, testing-skill of test engineers, testing-coverage. From such reason, the two-dimensional software reliability measurement technologies enable us to conduct more feasible software reliability assessment than the one-dimensional (conventional) software reliability measurement approach, in which it is assumed that the software reliability growth process depends only on testing-time. We discuss two-types of two-dimensional software reliability growth modeling approaches for feasible software reliability assessment, and conduct goodness-of-fit comparisons of our models with existing one-dimensional software reliability growth models. Finally, as one of the examples for two-dimensional software reliability analysis, we show examples of the application of our two-dimensional software reliability growth model by using actual data.

With the growth of complexity in the software system, to deliver reliable software products on time becomes a critical issue. Many software reliability growth models (SRGMs) have been proposed in the past three decades. However, most software reliability assessment processes and parameter estimations of models depend on the computations of the general-purposed numerical software. In this paper, we will present a powerful computer-aided reliability assessment tool for software based on object-oriented analysis and design-CARATS. CARATS can use both traditional SRGMs and neural-network methods to assess software reliability. This would greatly help project managers to make decisions during software development life cycle. Due to the characteristics of the special-purposed and object-oriented design, CARATS can analyze the software reliability easily, and is also more flexible to adopt different SRGMs than other existing software reliability analysis tools.

Many software reliability growth models have been proposed in the past decade. Those models tacitly assume that testing-effort expenditures are constant throughout software testing. This paper develops realistic software reliability growth models incorporating the effect of testing-effort. The software error detection phenomenon in software testing is modeled by a nonhomogeneous Poisson process. The software reliability assessment measures and the estimation methods of parameters are investigated. Testing-effort expenditures are described by exponential and Rayleigh curves. Least-squares estimators and maximum likelihood estimators are used for the reliability growth parameters. The software reliability data analyses use actual data. The software reliability growth models with testing-effort can consider the relationship between the software reliability growth and the effect of testing-effort. Thus, the proposed models will enable us to evaluate software reliability more realistically.

In reality, the fault detection (or correction) phenomenon and software reliability estimation in the operational phase are different from that in the testing phase. The fault removal continues at a slower rate during the operational phase. In this paper, we will investigate some techniques for software reliability prediction and measurement in the operational phase. We first review how some software reliability growth models (SRGMs) based on non-homogeneous Poisson processes (NHPPs) can be readily derived based on a unified theory for NHPP models. Under this general framework, we can not only verify some conventional SRGMs but also derive some new SRGMs that can be used for software reliability measurement in the operational phase. That is, based on the unified theory, we can incorporate the concept of multiple changepoints into software reliability modeling. We can formularize and simulate the fault detection process in the operational phase. Some numerical illustrations based on real software failure data are also presented. The experimental results show that the proposed models can easily reflect the possible changes of fault detection process and offer quantitative analysis on software failure behavior in field operation.

PurposeThe aim of this paper is to propose a software reliability growth model based on stochastic differential equations for the integration testing phase of distributed development environment.Design/methodology/approachA client/server system (CSS), which is a new development method, has come into existence as a result of the progress of networking technology by UNIX systems. On the other hand, the effective testing method for distributed development environment has only a few presented. The method of software reliability assessment considering the interaction among software components in a distributed one is discussed.FindingsConventional software reliability growth models for system testing phase in distributed development environment have included many unknown parameters. Especially, the effective estimation method in terms of these unknown parameters, which means the proportion of the total testing‐load for the software component, has never been presented. This software reliability growth model can be easily applied in distributed software development, because the model has a simple structure.Practical implicationsThis model is very useful for software developers in terms of practical reliability assessment in the actual distributed development environment.Originality/valueThe method of software reliability assessment considering the interaction among software components in distributed development environment is proposed. Additionally, several numerical examples for the actual data are presented.

A software reliability growth model for imperfect debugging

Owing to release of software in multiple releases, code changes take place in software. Because of this added complexity in software, the testing team may be unable to correct the fault upon detection, leaving the actual fault to reside in the software, termed as imperfect debugging or there may be replacement of original fault by other fault, leading to error generation. Many other factors exist that affect the testing phase of software like strategies of testing, test cases, skill, efficiency, and learning of testing team. All these factors cannot be kept stable during the whole process of testing. They may change at any time moment causing the background processes to experience change, which is known as change‐point. Keeping all these critical testing environment factors under consideration, a new software reliability growth model has been proposed, which is derived from an non homogenous Poisson process (NHPP)based unified scheme for multi‐release two‐stage fault detection/observation and correction/removal software reliability models. The developed model is numerically illustrated on tandem data set for four releases.