Recent Developments in Software Reliability Modeling

It is one of the important issues in modern software development to make a highly reliable software system by efficient and economical testing. As one of these solutions, many companies have taken in testing-progress management and quality/reliability assessment based on software reliability growth models. Consequently, many software reliability growth models reflecting various development factors and operational environment of the software system have been proposed. Most of these models have assumed the software system as one domain. That is, the models have assumed that the set of the testing-paths influenced by execution of test-cases is extended with testing-progress and it expands to whole software system finally. However, a software system has the structure that the main-component calls plural sub-components ordinarily. Therefore, unless the main-component is tested, the sub-component called from it is not tested. In this chapter, we propose software reliability growth models based on such components' characteristics. Especially, these models reflecting the different testing-environment for composed components are formulated by nonhomogeneous Poisson processes. Furthermore, using fault-detection data observed in actual development projects, we show numerical examples of software reliability assessment and results of goodness-of-fit comparisons with existing software reliability growth models.

PurposeThe aim of this paper is to propose a software reliability growth model based on stochastic differential equations for the integration testing phase of distributed development environment.Design/methodology/approachA client/server system (CSS), which is a new development method, has come into existence as a result of the progress of networking technology by UNIX systems. On the other hand, the effective testing method for distributed development environment has only a few presented. The method of software reliability assessment considering the interaction among software components in a distributed one is discussed.FindingsConventional software reliability growth models for system testing phase in distributed development environment have included many unknown parameters. Especially, the effective estimation method in terms of these unknown parameters, which means the proportion of the total testing‐load for the software component, has never been presented. This software reliability growth model can be easily applied in distributed software development, because the model has a simple structure.Practical implicationsThis model is very useful for software developers in terms of practical reliability assessment in the actual distributed development environment.Originality/valueThe method of software reliability assessment considering the interaction among software components in distributed development environment is proposed. Additionally, several numerical examples for the actual data are presented.

Software reliability assessment for an exponential-S-shaped reliability growth phenomenon

In reality, the fault detection (or correction) phenomenon and software reliability estimation in the operational phase are different from that in the testing phase. The fault removal continues at a slower rate during the operational phase. In this paper, we will investigate some techniques for software reliability prediction and measurement in the operational phase. We first review how some software reliability growth models (SRGMs) based on non-homogeneous Poisson processes (NHPPs) can be readily derived based on a unified theory for NHPP models. Under this general framework, we can not only verify some conventional SRGMs but also derive some new SRGMs that can be used for software reliability measurement in the operational phase. That is, based on the unified theory, we can incorporate the concept of multiple changepoints into software reliability modeling. We can formularize and simulate the fault detection process in the operational phase. Some numerical illustrations based on real software failure data are also presented. The experimental results show that the proposed models can easily reflect the possible changes of fault detection process and offer quantitative analysis on software failure behavior in field operation.

The safety-critical software of Reactor Protection System (RPS) plays a significant role for the safe operation of the nuclear power plant (NPP). However, it also brings challenges both to the reliability analysis of the RPS and to the Probabilistic Risk Assessment of the NPPs. The reliability analysis of safety-critical software is also expected by the nuclear regulation agencies and the software development groups for test evaluation and optimization. The detected faults during the software test process are regarded to have close connection with the software reliability and there have been hundreds of test-based software reliability models. Due to the particularity of its function, the safety-critical software of an NPP is especially sensitive to the faults with higher severity levels which should be paid special attention to. Severity levels of faults are commonly taken into account during software reliability modelling. In this paper, a novel software reliability growth model considering actual severity data was built based on a non-homogeneous Poisson process. Ratio of fatal faults with highest severity level to all accumulated faults was modelled with logistic curve. Then the mean value functions of both fatal and general faults were derived. A belief factor was employed to describe the trend of severity classification. In the end, the test data of a practical project was used to verify this model. The analysis results showed that this model had better prediction effect than Goel-Okumoto model and Inflection S-shaped model even when limited test data were collected. This novel model can be used as a helpful tool to evaluate both the reliability and the release time of the safety-critical software of an NPP.

In order to overcome the limitations of debugging process, insufficient consideration of imperfect debugging and testing-effort (TE) in software reliability modeling and analysis, a software reliability growth model (SRGM) explicitly incorporating imperfect debugging and TE is developed. From the point of view of incomplete debugging and introduction of new fault, software testing process is described and a relatively unified SRGM framework is presented considering TE. The proposed framework models are fairly general models that cover a variety of the previous works on SRGM with ID and TE. Furthermore, a special SRGM incorporating an improved Logistic testing-effort function (TEF) into imperfect debugging modeling is proposed. The effectiveness and reasonableness of the proposed model are verified by published failure data set. The proposed model closer to real software testing has better descriptive and predictive power than other models

Performance analysis of software reliability growth models with testing-effort and change-point

In the software testing process, many factors can affected the failure distribution, such as the testing strategy, running environment and resource allocation, etc. Once these factors are transformed during testing phase, they could result in the failure intensity function increasing or decreasing non-monotonically and abrupt fluctuations at a certain time interval. Especially, in large software systems, detected complex faults and failures are more difficult to isolate, reproduce and remove in comparison of detected simple faults and failures. Therefore, the fault detection rate shows the irregular fluctuation over time in the software testing process. In this paper, we propose an imperfect software debugging model derived from an Ito -type stochastic differential equation (SDE). In the actual testing process, the cumulative number of fault detection can be considered a stochastic event. Thus, we can model fault detection process using SDE. We estimate the parameters of the proposed model using maximum likelihood estimation. We also use five historical fault data sets to test the performance of the proposed model. To show that the proposed model is the best model among all models, we use a new optimal method to select the best model from various software reliability models. Experimental results show that the proposed model has better fitting and predictive performance than non-homogeneous Poisson process models and other perfect software reliability models based on SDE.

A large number of software reliability growth models (SRGMs) have been proposed during the past 30 years to estimate software reliability measures such as the number of remaining faults, software failure rate, and software reliability. Selection of an optimal SRGM for use in a particular case has been an area of interest for researchers in the field of software reliability. Tools and techniques for software reliability model selection found in the literature cannot be used with high confidence as they use a limited number of model selection criteria. For the first time, we developed a deterministic quantitative model based on a distance based approach (DBA) method, then applied it for evaluation, optimal selection, and ranking of SRGMs. DBA recognizes the need for relative importance of criteria for a given application, without which inter-criterion comparison could not be accomplished. It requires a set of model selection criteria, along with a set of SRGMs, and their level of criteria for optimal selection; and it successfully presents the results in terms of a merit value which is used to rank the SRGMs. We use two distinct, real data sets for demonstration of the DBA method. The result of this study will be a ranking of SRGMs based on the Euclidean composite distance of each alternative to the designated optimal SRGM.

With the growth of complexity in the software system, to deliver reliable software products on time becomes a critical issue. Many software reliability growth models (SRGMs) have been proposed in the past three decades. However, most software reliability assessment processes and parameter estimations of models depend on the computations of the general-purposed numerical software. In this paper, we will present a powerful computer-aided reliability assessment tool for software based on object-oriented analysis and design-CARATS. CARATS can use both traditional SRGMs and neural-network methods to assess software reliability. This would greatly help project managers to make decisions during software development life cycle. Due to the characteristics of the special-purposed and object-oriented design, CARATS can analyze the software reliability easily, and is also more flexible to adopt different SRGMs than other existing software reliability analysis tools.

One of the established trends of research areas and practices in software engineering that dealt with the measurement and enhancement of reliability is software reliability engineering. Stochastic software reliability models find typical usage for analysis. These are the models that perform modeling on failure process of the software and exploit other software metrics or failure data as cornerstone for parameter estimation. The ability of the models for estimating and predicting the current reliability and future failure behavior, respectively, is high. Due to any failures and faults in the system, a product becomes unreliable. The lack of understanding of nature of the software makes the measurement of software reliability as a challenging task. It is not possible to determine a best way to measure the reliability and other aspects of software. This paper proposes an efficient software reliability growth model (SRGM) in which logistic exponential TEF is exploited. This model offers increased failure rate recognition and suitable ways to resolve faults and so on. Our work estimates the SRGM parameters using optimization algorithm. Such estimation can aid in developing precise software reliability model. In order to accomplish the optimization, we use artificial bee colony (ABC) algorithm. As the parameters optimization considerably improves the quality of parameters to be used for reliability growth model, reliability growth can also be improved considerably.

There are many models of software reliability growth, but none of them is able to model the varied patterns observed in practice. A previous paper (R.E. Mullen, 1998), suggested that since event rates in software systems are generated by multiplicative processes, the distribution of rates of events, including failure rates, is lognormal. It also showed that many previously published empirical failure rate distributions are well fit by the lognormal. The theoretical roots and experimental confirmation of the lognormal distribution of failure rates in software systems provide a unique and potentially fruitful basis for constructing a new software reliability model. The paper derives a Lognormal Execution Time Software Reliability Growth Model, a member of the family of Doubly Stochastic Exponential Order Statistic Models, by developing a numerical approximation for the Laplace transform of the lognormal. The model is used to analyze two series of failure data as an example of its use. The likelihood of that data arising from the lognormal and log-Poisson models is computed and shown to be highly favorable to the lognormal in one case and slightly favorable in the other. A preliminary comparison of the lognormal, the LPET, and the BET models using ten Musa data sets further demonstrates the ability of the lognormal model to fit a wide variety of reliability growth scenarios. Of particular novelty is the use of a software failure rate model which has both plausible theoretical justification and solid support from prior studies of software failure rate distributions. Also novel is the application of the Laplace Transform of the Lognormal to the problem of software reliability growth.

Software testing is an essential activity in all software projects. The key issue in testing is determining the sufficiency of tests and, traditionally this has been done by using the Software Reliability Growth Models (SRGMs). However, SRGMs are not applicable when there are no stabilization phases as required by most software reliability models. Test-Defect Coverage Analytics Model (TDCAM) has been proposed to address this problem. This paper proposes the application of visual analytic techniques as an approach for supporting informed decision making in deciding the sufficiency of tests. Visual analytics research considers interactive visualization as the common platform for combining various computational data analysis techniques to support the analytical reasoning process. We presented four visual representations of TDCAM to demonstrate how analytical models have been applied to indicate the adequacy of tests in relation to sufficient and efficient test coverage. The techniques provide an effective and generally applicable test estimation on the basis of a general trend that higher test coverage correlates with higher probability of detecting more defects.

In current software reliability modeling research, the main concern is how to develop general prediction models. In this paper, we propose several improvements on the conventional software reliability growth models (SRGMs) to describe actual software development process by eliminating some unrealistic assumptions. Most of these models have focused on the failure detection process and not given equal priority to modeling the fault correction process. But, most latent software errors may remain uncorrected for a long time even after they are detected, which increases their impact. The remaining software faults are often one of the most unreliable reasons for software quality. Therefore, we develop a general framework of the modeling of the failure detection and fault correction processes. Furthermore, we apply neural network with back-propagation to match the histories of software failure data. We will also illustrate how to construct the neural networks from the mathematical viewpoints of software reliability modeling in detail. Finally, numerical examples are shown to illustrate the results of the integration of the detection and correction process in terms of predictive ability and some other standard criteria.

Software reliability is deliberated as a measurable metric, which is the probability of any software operation to be free of failure for a stated course of time in a given environment. Many Software Reliability Growth Models have been developed over the years that can calculate and anticipate the software product reliability. This paper gives an analysis of various Soft Computing Techniques and considers these soft computing techniques in terms of software reliability modeling competence.