Queueing-Based Simulation for Software Reliability Analysis

Over the past 30 years, many software reliability growth models (SRGMs) have been proposed for estimation of reliability growth of software. In fact, effective debugging will not be easy because the fault may not be immediately obvious. In the past, some researchers ever used an infinite server queueing (ISQ) model to describe the software debugging behavior. An infinite-server queueing model is considered where access of customers to service is controlled by a gate and the gate is open only if all servers are free. However, the finite server queueing (FSQ) model is first advantageously modeled as an infinite-server system. Thus, in this paper, we will show how to incorporate both FSQ and ISQ models into software reliability estimation and prediction. In addition, here we also consider the factor of perfect/imperfect debugging. Experimental results show that the proposed framework to incorporate both fault detection and correction processes for SRGM has a fairly accurate prediction capability.

Software testing involves verification and validation of the software to meet the requirements elucidated by customers in the earlier phases and to subsequently increase software reliability. Around half of the resources, such as manpower and CPU time are consumed and a major portion of the total cost of developing the software is incurred in testing phase, making it the most crucial and time-consuming phase of a software development lifecycle (SDLC). Also the fault detection process (FDP) and fault correction process (FCP) are the important processes in SDLC. A number of software reliability growth models (SRGM) have been proposed in the last four decades to capture the time lag between detected and corrected faults. But most of the models are discussed under static environment. The purpose of this paper is to allocate the resources in an optimal manner to minimize the cost during testing phase using FDP and FCP under dynamic environment. An elaborate optimization policy based on optimal control theory for resource allocation with the objective to minimize the cost is proposed. Further, genetic algorithm is applied to obtain the optimum value of detection and correction efforts which minimizes the cost. Numerical example is given in support of the above theoretical result. The experimental results help the project manager to identify the contribution of model parameters and their weight.

The fault correction process is very important in software testing, and it has been considered into some software reliability growth models (SRGMs). In these models, the time-delay functions are often used to describe the dependency of the fault detection and correction processes. In this paper, a more direct variable "correction lag", which is defined as the difference between the detected and corrected fault numbers, is addressed to characterize the dependency of the two processes. We investigate the correction lag and find that it appears Bell-shaped. Therefore, we adopt the Gamma function to describe the correction lag. Based on this function, a new SRGM which includes the fault correction process is proposed. And the experimental results show that the new model gives better fit and prediction than other models.

Software reliability can be enhanced considerably during testing with faults being detected and corrected by testers. The allocation of testing resources, such as man power and CPU hours, during testing phase can largely influence fault detection speed and the time to correct a detected fault. The testing resources allocation is usually depicted by testing effort function, which has been incorporated into software reliability models in some recent papers. Fault correction process (FCP) is usually modeled as a delayed process of fault detection process (FDP). In addition, debugging is usually not perfect and new faults can be introduced during testing. In this paper, flexible testing effort dependent paired models of FDP and FCP are derived with consideration of fault introduction. A real dataset is used to illustrate the application of proposed models.

Over the past three decades, many software reliability growth models (SRGMs) were proposed and they are aimed at predicting and estimating software reliability. One common assumption of these conventional SRGMs is that detected faults will be removed immediately. In reality, this assumption may not be reasonable and may not always occur. Developers need time to identify the root causes of detected faults and then fix them. Besides, during debugging the fault correction rate may not be a constant and could be changed at some certain points as time proceeds. Consequently, in this paper, we will explore and study how to apply queueing model to investigate the fault correction process during software development. We propose an extended infinite server queueing model with multiple change-points to predict and assess software reliability. Experimental results based on real failure data show that proposed model can depicts the change of fault correction rates and predict the behavior of software development more accurately than traditional SRGMs.

In current software reliability modeling research, the main concern is how to develop general prediction models. In this paper, we propose several improvements on the conventional software reliability growth models (SRGMs) to describe actual software development process by eliminating some unrealistic assumptions. Most of these models have focused on the failure detection process and not given equal priority to modeling the fault correction process. But, most latent software errors may remain uncorrected for a long time even after they are detected, which increases their impact. The remaining software faults are often one of the most unreliable reasons for software quality. Therefore, we develop a general framework of the modeling of the failure detection and fault correction processes. Furthermore, we apply neural network with back-propagation to match the histories of software failure data. We will also illustrate how to construct the neural networks from the mathematical viewpoints of software reliability modeling in detail. Finally, numerical examples are shown to illustrate the results of the integration of the detection and correction process in terms of predictive ability and some other standard criteria.

In reality, the fault detection (or correction) phenomenon and software reliability estimation in the operational phase are different from that in the testing phase. The fault removal continues at a slower rate during the operational phase. In this paper, we will investigate some techniques for software reliability prediction and measurement in the operational phase. We first review how some software reliability growth models (SRGMs) based on non-homogeneous Poisson processes (NHPPs) can be readily derived based on a unified theory for NHPP models. Under this general framework, we can not only verify some conventional SRGMs but also derive some new SRGMs that can be used for software reliability measurement in the operational phase. That is, based on the unified theory, we can incorporate the concept of multiple changepoints into software reliability modeling. We can formularize and simulate the fault detection process in the operational phase. Some numerical illustrations based on real software failure data are also presented. The experimental results show that the proposed models can easily reflect the possible changes of fault detection process and offer quantitative analysis on software failure behavior in field operation.

A large number of software reliability growth models (SRGMs) have been proposed during the past 30 years to estimate software reliability measures such as the number of remaining faults, software failure rate, and software reliability. Selection of an optimal SRGM for use in a particular case has been an area of interest for researchers in the field of software reliability. Tools and techniques for software reliability model selection found in the literature cannot be used with high confidence as they use a limited number of model selection criteria. For the first time, we developed a deterministic quantitative model based on a distance based approach (DBA) method, then applied it for evaluation, optimal selection, and ranking of SRGMs. DBA recognizes the need for relative importance of criteria for a given application, without which inter-criterion comparison could not be accomplished. It requires a set of model selection criteria, along with a set of SRGMs, and their level of criteria for optimal selection; and it successfully presents the results in terms of a merit value which is used to rank the SRGMs. We use two distinct, real data sets for demonstration of the DBA method. The result of this study will be a ranking of SRGMs based on the Euclidean composite distance of each alternative to the designated optimal SRGM.

With the growth of complexity in the software system, to deliver reliable software products on time becomes a critical issue. Many software reliability growth models (SRGMs) have been proposed in the past three decades. However, most software reliability assessment processes and parameter estimations of models depend on the computations of the general-purposed numerical software. In this paper, we will present a powerful computer-aided reliability assessment tool for software based on object-oriented analysis and design-CARATS. CARATS can use both traditional SRGMs and neural-network methods to assess software reliability. This would greatly help project managers to make decisions during software development life cycle. Due to the characteristics of the special-purposed and object-oriented design, CARATS can analyze the software reliability easily, and is also more flexible to adopt different SRGMs than other existing software reliability analysis tools.

Many software reliability growth models have been proposed in the past decade. Those models tacitly assume that testing-effort expenditures are constant throughout software testing. This paper develops realistic software reliability growth models incorporating the effect of testing-effort. The software error detection phenomenon in software testing is modeled by a nonhomogeneous Poisson process. The software reliability assessment measures and the estimation methods of parameters are investigated. Testing-effort expenditures are described by exponential and Rayleigh curves. Least-squares estimators and maximum likelihood estimators are used for the reliability growth parameters. The software reliability data analyses use actual data. The software reliability growth models with testing-effort can consider the relationship between the software reliability growth and the effect of testing-effort. Thus, the proposed models will enable us to evaluate software reliability more realistically.

With the growing competition and the demand of the customers, a software organization needs to regularly provide up-gradations and add features to its existing version of software. For the organization, creating these software upgrades means an increase in the complexity of the software which in turn leads to the increase in the number of faults. Also, the faults left undetected in the previous version need to be addressed in this phase. Many software reliability growth models have been proposed to model the phenomenon of multi-release problems using two stage failure observation and correction processes. The model proposed in this paper partitions the fault removal process into a two-stage process which includes fault detection process and fault removal process considering the joint effect of premeditated release pressure and resource restrictions using a well-known Cobb–Douglas production function for the multi release problem of a software. The faults detected in the operational phase of the previous release or left incomplete are also incorporated in the next release. A generalized framework for the multi-release problem in which fault detection follows an exponential distribution function and fault correction follows Gamma distribution function is proposed and verified on a real data set of four releases of software. The estimated parameters and comparison criteria are also given.

In order to incorporate the effect of test coverage, two novel software reliability growth models (SRGMs) are proposed in this paper using failure data and test coverage simultaneously. One is continuous using testing time, and the other is discrete with respect to the number of executed test cases instead of testing time. Since one of the most important factors of the coverage-based SRGMs is the test coverage function (TCF), we first discuss a discrete TCF based on Beta function. Then we develop mean value functions (MVF) of the two models integrating test coverage and imperfect debugging. Finally the proposed TCF and MVFs are evaluated and validated on actual software reliability data collected from real software development projects. The results demonstrate clearly that both the proposed TCF and SRGMs provide better estimation and fitting for the data sets under comparisons.

It is one of the important issues in modern software development to make a highly reliable software system by efficient and economical testing. As one of these solutions, many companies have taken in testing-progress management and quality/reliability assessment based on software reliability growth models. Consequently, many software reliability growth models reflecting various development factors and operational environment of the software system have been proposed. Most of these models have assumed the software system as one domain. That is, the models have assumed that the set of the testing-paths influenced by execution of test-cases is extended with testing-progress and it expands to whole software system finally. However, a software system has the structure that the main-component calls plural sub-components ordinarily. Therefore, unless the main-component is tested, the sub-component called from it is not tested. In this chapter, we propose software reliability growth models based on such components' characteristics. Especially, these models reflecting the different testing-environment for composed components are formulated by nonhomogeneous Poisson processes. Furthermore, using fault-detection data observed in actual development projects, we show numerical examples of software reliability assessment and results of goodness-of-fit comparisons with existing software reliability growth models.

PurposeThe aim of this paper is to propose a software reliability growth model based on stochastic differential equations for the integration testing phase of distributed development environment.Design/methodology/approachA client/server system (CSS), which is a new development method, has come into existence as a result of the progress of networking technology by UNIX systems. On the other hand, the effective testing method for distributed development environment has only a few presented. The method of software reliability assessment considering the interaction among software components in a distributed one is discussed.FindingsConventional software reliability growth models for system testing phase in distributed development environment have included many unknown parameters. Especially, the effective estimation method in terms of these unknown parameters, which means the proportion of the total testing‐load for the software component, has never been presented. This software reliability growth model can be easily applied in distributed software development, because the model has a simple structure.Practical implicationsThis model is very useful for software developers in terms of practical reliability assessment in the actual distributed development environment.Originality/valueThe method of software reliability assessment considering the interaction among software components in distributed development environment is proposed. Additionally, several numerical examples for the actual data are presented.

In order to overcome the limitations of debugging process, insufficient consideration of imperfect debugging and testing-effort (TE) in software reliability modeling and analysis, a software reliability growth model (SRGM) explicitly incorporating imperfect debugging and TE is developed. From the point of view of incomplete debugging and introduction of new fault, software testing process is described and a relatively unified SRGM framework is presented considering TE. The proposed framework models are fairly general models that cover a variety of the previous works on SRGM with ID and TE. Furthermore, a special SRGM incorporating an improved Logistic testing-effort function (TEF) into imperfect debugging modeling is proposed. The effectiveness and reasonableness of the proposed model are verified by published failure data set. The proposed model closer to real software testing has better descriptive and predictive power than other models