Incorporating fault dependency and debugging delay in software reliability analysis

Over the past three decades, many software reliability growth models (SRGMs) were proposed and they are aimed at predicting and estimating software reliability. One common assumption of these conventional SRGMs is that detected faults will be removed immediately. In reality, this assumption may not be reasonable and may not always occur. Developers need time to identify the root causes of detected faults and then fix them. Besides, during debugging the fault correction rate may not be a constant and could be changed at some certain points as time proceeds. Consequently, in this paper, we will explore and study how to apply queueing model to investigate the fault correction process during software development. We propose an extended infinite server queueing model with multiple change-points to predict and assess software reliability. Experimental results based on real failure data show that proposed model can depicts the change of fault correction rates and predict the behavior of software development more accurately than traditional SRGMs.

With the growth of complexity in the software system, to deliver reliable software products on time becomes a critical issue. Many software reliability growth models (SRGMs) have been proposed in the past three decades. However, most software reliability assessment processes and parameter estimations of models depend on the computations of the general-purposed numerical software. In this paper, we will present a powerful computer-aided reliability assessment tool for software based on object-oriented analysis and design-CARATS. CARATS can use both traditional SRGMs and neural-network methods to assess software reliability. This would greatly help project managers to make decisions during software development life cycle. Due to the characteristics of the special-purposed and object-oriented design, CARATS can analyze the software reliability easily, and is also more flexible to adopt different SRGMs than other existing software reliability analysis tools.

Reliability of software is the key factor of software quality estimation during the testing period of software. This paper proposes a nonparametric method using radial basis function neural network for predicting software reliability. The Bayesian Regularization method is applied in the proposed model to improve the generalization and to avoid the over fitting problem. The proposed scheme has been tested on five benchmark datasets. The results of the system are compared with other states of the traditional method and it is observed that the proposed architecture outperforms its competent systems. The results of the proposed architecture have been adequately tested with a single feed-forward neural network model and a linear parametric software reliability growth model. The experimental result shows that the radial basis function of neural network yields better performance than the traditional software reliability growth model (SRGM).

This paper proposes a new scheme for constructing software reliability growth models (SRGM) based on a nonhomogeneous Poisson process (NHPP). The main focus is to provide an efficient parametric decomposition method for software reliability modeling, which considers both testing efforts and fault detection rates (FDR). In general, the software fault detection/removal mechanisms depend on previously detected/removed faults and on how testing efforts are used. From practical field studies, it is likely that we can estimate the testing efforts consumption pattern and predict the trends of FDR. A set of time-variable, testing-effort-based FDR models were developed that have the inherent flexibility of capturing a wide range of possible fault detection trends: increasing, decreasing, and constant. This scheme has a flexible structure and can model a wide spectrum of software development environments, considering various testing efforts. The paper describes the FDR, which can be obtained from historical records of previous releases or other similar software projects, and incorporates the related testing activities into this new modeling approach. The applicability of our model and the related parametric decomposition methods are demonstrated through several real data sets from various software projects. The evaluation results show that the proposed framework to incorporate testing efforts and FDR for SRGM has a fairly accurate prediction capability and it depicts the real-life situation more faithfully. This technique can be applied to wide range of software systems.

In reality, the fault detection (or correction) phenomenon and software reliability estimation in the operational phase are different from that in the testing phase. The fault removal continues at a slower rate during the operational phase. In this paper, we will investigate some techniques for software reliability prediction and measurement in the operational phase. We first review how some software reliability growth models (SRGMs) based on non-homogeneous Poisson processes (NHPPs) can be readily derived based on a unified theory for NHPP models. Under this general framework, we can not only verify some conventional SRGMs but also derive some new SRGMs that can be used for software reliability measurement in the operational phase. That is, based on the unified theory, we can incorporate the concept of multiple changepoints into software reliability modeling. We can formularize and simulate the fault detection process in the operational phase. Some numerical illustrations based on real software failure data are also presented. The experimental results show that the proposed models can easily reflect the possible changes of fault detection process and offer quantitative analysis on software failure behavior in field operation.

To date, over 75 software reliability models have appeared in the literature These models, however, require sophisticated numerical procedures to obtain estimates of the model parameters. This necessitates the use of a computer. This was the purpose of the development of the program called SMERFS (Statistical Modeling and Estimation of Reliability Functions for Software). It is an interactive program for software reliability modeling that provided a computer-based tool for the software analyst to perform software reliability analyses. That program has undergone a number of revisions over the years. SMERFS/sup /spl and//3 is the newest evolution of SMERFS. It is a strictly PC Windows-based application program that allows the user to carry out hardware, software and/or total systems reliability analyses. The aim is to perform both component- and systems-level assessments.

Software reliability is generally a key factor in software quality. Reliability is an essential ingredient in customer satisfaction. In software development process reliability conveys the information to managers to access the testing effort and time at which software release into the market. Large numbers of papers are published in this context. In this paper we proposed a software reliability growth model with generalized modified weibull testing effort. Performance application of proposed model is demonstrated through real datasets. The experimental results shown that the model gives an excellent performance compared to other models. We also discuss the optimal release time based on reliability requirement and cost criteria.

The safety-critical software of Reactor Protection System (RPS) plays a significant role for the safe operation of the nuclear power plant (NPP). However, it also brings challenges both to the reliability analysis of the RPS and to the Probabilistic Risk Assessment of the NPPs. The reliability analysis of safety-critical software is also expected by the nuclear regulation agencies and the software development groups for test evaluation and optimization. The detected faults during the software test process are regarded to have close connection with the software reliability and there have been hundreds of test-based software reliability models. Due to the particularity of its function, the safety-critical software of an NPP is especially sensitive to the faults with higher severity levels which should be paid special attention to. Severity levels of faults are commonly taken into account during software reliability modelling. In this paper, a novel software reliability growth model considering actual severity data was built based on a non-homogeneous Poisson process. Ratio of fatal faults with highest severity level to all accumulated faults was modelled with logistic curve. Then the mean value functions of both fatal and general faults were derived. A belief factor was employed to describe the trend of severity classification. In the end, the test data of a practical project was used to verify this model. The analysis results showed that this model had better prediction effect than Goel-Okumoto model and Inflection S-shaped model even when limited test data were collected. This novel model can be used as a helpful tool to evaluate both the reliability and the release time of the safety-critical software of an NPP.

This paper discusses two-dimensional software reliability measurement technologies, which describe a software reliability growth process depending on two-types of software reliability growth factors: Testing-time and testing-effort factors. From the view point of actual software failure-occurrence mechanism, it is natural to consider that a software reliability growth process depends on not only testing-time factor but also other software reliability growth factors, i.e., testing-effort factors, such as test-execution time, testing-skill of test engineers, testing-coverage. From such reason, the two-dimensional software reliability measurement technologies enable us to conduct more feasible software reliability assessment than the one-dimensional (conventional) software reliability measurement approach, in which it is assumed that the software reliability growth process depends only on testing-time. We discuss two-types of two-dimensional software reliability growth modeling approaches for feasible software reliability assessment, and conduct goodness-of-fit comparisons of our models with existing one-dimensional software reliability growth models. Finally, as one of the examples for two-dimensional software reliability analysis, we show examples of the application of our two-dimensional software reliability growth model by using actual data.

SUMMARY & CONCLUSIONSThis paper encourages the reader to consider different approaches for characterizing and tracking software Reliability and Availability (R&A) metrics, where reliability is a design characteristic measure of mission performance, and availability is a performance characteristic of operational capability. The primary use case examined in this paper considers large, complex distributed satellite ground stations developed using test-driven methodologies, e.g. Agile. In both cases, whether generating reliability or availability metrics, it can be quite difficult to accurately predict performance without having key data sets. Considering Atlassian’s Jira™ as an industry preferred tool for Agile development, the desire is to automate calculations using data sets from Jira™. However, the default template that Jira™ provides to document faults is inadequate for developing such an automated process without modification, i.e. formalizing as many fields as possible with dropdown menus. This simple modification enables various R&A metrics to be automated based on real data. In turn, automation helps develop fault densities, Mean Time Between Failures (MTBF), Availability (Ao), and other desired metrics.To ensure metrics are accurately and efficiently generated, it is necessary to develop and implement automation techniques in a way that aligns with software reliability activities, i.e. IEEE-1633 [1]. Failure Reporting Analysis and Corrective Action (FRACA) systems are rich data sources that can be exploited in this manner, e.g. software defects, defect density, etc. Therefore, it is recommended that test-driven methodologies and tools are adapted as soon as practical so R&A attributes can be inherently integrated into the fabric of the system design and architecture. Doing so allows for a comprehensive dataset that is needed to generate essential metrics. Once the framework is created, sample data sets are used to demonstrate proof of concepts. In summary, generating automated calculation of R&A metrics, utilizing Jira™ data, drives efficiency into the various activities and enhances the fidelity of the generated metrics. This paper walks through the framework and recommendations to implement into Jira™, the process behind utilizing that information, and results from a proof of concept.

In order to overcome the limitations of debugging process, insufficient consideration of imperfect debugging and testing-effort (TE) in software reliability modeling and analysis, a software reliability growth model (SRGM) explicitly incorporating imperfect debugging and TE is developed. From the point of view of incomplete debugging and introduction of new fault, software testing process is described and a relatively unified SRGM framework is presented considering TE. The proposed framework models are fairly general models that cover a variety of the previous works on SRGM with ID and TE. Furthermore, a special SRGM incorporating an improved Logistic testing-effort function (TEF) into imperfect debugging modeling is proposed. The effectiveness and reasonableness of the proposed model are verified by published failure data set. The proposed model closer to real software testing has better descriptive and predictive power than other models

A large number of software reliability growth models (SRGMs) have been proposed during the past 30 years to estimate software reliability measures such as the number of remaining faults, software failure rate, and software reliability. Selection of an optimal SRGM for use in a particular case has been an area of interest for researchers in the field of software reliability. Tools and techniques for software reliability model selection found in the literature cannot be used with high confidence as they use a limited number of model selection criteria. For the first time, we developed a deterministic quantitative model based on a distance based approach (DBA) method, then applied it for evaluation, optimal selection, and ranking of SRGMs. DBA recognizes the need for relative importance of criteria for a given application, without which inter-criterion comparison could not be accomplished. It requires a set of model selection criteria, along with a set of SRGMs, and their level of criteria for optimal selection; and it successfully presents the results in terms of a merit value which is used to rank the SRGMs. We use two distinct, real data sets for demonstration of the DBA method. The result of this study will be a ranking of SRGMs based on the Euclidean composite distance of each alternative to the designated optimal SRGM.

In this paper, we propose an artificial neural- network-based approach for software reliability estimation and modeling. We first explain the network networks from the mathematical viewpoints of software reliability modeling. That is, we will show how to apply neural network to predict software reliability by designing different elements of neural networks. Furthermore, we will use the neural network approach to build a dynamic weighted combinational model. The applicability of proposed model is demonstrated through four real software failure data sets. From experimental results, we can see that the proposed model significantly outperforms the traditional software reliability models.

The prevalence of software as an integral part of almost anything is commonplace today among the products, systems, applications, services and/or solutions with which we all interact. Software reliability directly impacts both the user experience and field operating costs. A key challenge is for the product developer to accurately predict and improve software field reliability in terms of attributes such as outage frequency and duration, prior to software delivery. Unfortunately, the actual field performance data does not often correlate with the predictions. Markov analysis and software reliability growth models are necessary key steps, but are insufficient to fully address customer-perceived reliability issues. Can we make better use of field performance data to improve our predictive models and tune our test strategies from one release to the next? In this paper, we describe an integrative software reliability framework that attempts to close the gap between expectations, predictions, and the actual behavior of systems by leveraging key best-in-class practices, tools, and techniques for software reliability analysis, modeling and predicting software field performance in terms of outage frequency/duration; tracking and monitoring test defect data and defects “not fixed” in subsequent releases over the software lifecycle, and validating prediction results with the actual field data. The proposed approach is designed to achieve customer expectations for software reliability and help assure the delivery of highly reliable software products. © 2012 Alcatel-Lucent.

Software reliability growth models based on a nonhomogeneous Poisson process (NHPP) have been considered as one of the most effective among various models since they integrate the information regarding testing and debugging activities observed in the testing phase into the software reliability model. Although most of the existing NHPP models have progressed successfully in their estimation/prediction accuracies by modifying the assumptions with regard to the testing process, these models were developed based on the instantaneous fault-removal assumption. In this paper, we develop a generalized NHPP software reliability model considering quasi-renewal time-delay fault removal. The quasi-renewal process is employed to estimate the time delay due to identifying and prioritizing the detected faults before actual code change in the software reliability assessment. Model formulation based on the quasi-renewal time-delay assumption is provided, and the generalized mean value function (MVF) for the proposed model is derived by using the method of steps. The general solution of the MVFs for the proposed model is also obtained for some specific existing models. The numerical examples, based on a software failure data set, show that the consideration of quasi-renewal time-delay fault-removal assumption improves the descriptive properties of the model, which means that the length of time delay is getting decreased since testers and programmers adapt themselves to the working environment as testing and debugging activities are in progress.