Dynamic Performance Analysis for Software System Considering Real-Time Property in Case of NHPP Task Arrival

This paper constructs the integrated software performability model reflecting the actual field environment. The time-dependent behavior of the system alternating between up and down states is described by the Markov process. Then we incorporate the undesirable operational state and the field-oriented restoration scenario into the model, i.e., we consider the following two types of operational state: one is operating with the desirable performance level according to specification and the other is with degraded performance level, and the following two types of restoration: one is the restoration with debugging and the other is without debugging. Assuming that the software system can process the multiple tasks simultaneously, we describe the arrival process of the tasks follows a nonhomogeneous Poisson process and treat the processing time limit imposed on each task as a random variable. We analyze the distribution of the number of tasks whose processes can be completed within the processing time limit with the infinite server queueing model. From the model, we derive several software performability measures considering the real-time property. Finally, we illustrate several numerical examples of the measures to investigate the impacts of the difference of the operational performance levels and the field-oriented restoration on the system performability evaluation.

Over the past 30 years, many software reliability growth models (SRGMs) have been proposed for estimation of reliability growth of software. In fact, effective debugging will not be easy because the fault may not be immediately obvious. In the past, some researchers ever used an infinite server queueing (ISQ) model to describe the software debugging behavior. An infinite-server queueing model is considered where access of customers to service is controlled by a gate and the gate is open only if all servers are free. However, the finite server queueing (FSQ) model is first advantageously modeled as an infinite-server system. Thus, in this paper, we will show how to incorporate both FSQ and ISQ models into software reliability estimation and prediction. In addition, here we also consider the factor of perfect/imperfect debugging. Experimental results show that the proposed framework to incorporate both fault detection and correction processes for SRGM has a fairly accurate prediction capability.

Software reliability models are used to estimate and predict the reliability, number of remaining faults, failure intensity, total software development cost, etc., of a software. Testing coverage is very important for both software developers and customers of software products. Testing converge is a measure that enables software developers to evaluate the quality of tested software and determine how much additional effort is needed to improve the reliability of the software. This paper proposes a software reliability growth model based on a non-homogeneous Poisson process (NHPP) that incorporates a logistic–exponential testing coverage function with imperfect debugging. The proposed model relates the test coverage to fault detection phenomena in debugging. Goodness-of-fit test of the proposed model is conducted using different criteria for two sets of software failure data. The proposed model is compared with other existing NHPP models. A software cost model incorporating testing coverage and an optimal release policy based on the number of remaining faults are developed.

In the software testing process, many factors can affected the failure distribution, such as the testing strategy, running environment and resource allocation, etc. Once these factors are transformed during testing phase, they could result in the failure intensity function increasing or decreasing non-monotonically and abrupt fluctuations at a certain time interval. Especially, in large software systems, detected complex faults and failures are more difficult to isolate, reproduce and remove in comparison of detected simple faults and failures. Therefore, the fault detection rate shows the irregular fluctuation over time in the software testing process. In this paper, we propose an imperfect software debugging model derived from an Ito -type stochastic differential equation (SDE). In the actual testing process, the cumulative number of fault detection can be considered a stochastic event. Thus, we can model fault detection process using SDE. We estimate the parameters of the proposed model using maximum likelihood estimation. We also use five historical fault data sets to test the performance of the proposed model. To show that the proposed model is the best model among all models, we use a new optimal method to select the best model from various software reliability models. Experimental results show that the proposed model has better fitting and predictive performance than non-homogeneous Poisson process models and other perfect software reliability models based on SDE.

Owing to release of software in multiple releases, code changes take place in software. Because of this added complexity in software, the testing team may be unable to correct the fault upon detection, leaving the actual fault to reside in the software, termed as imperfect debugging or there may be replacement of original fault by other fault, leading to error generation. Many other factors exist that affect the testing phase of software like strategies of testing, test cases, skill, efficiency, and learning of testing team. All these factors cannot be kept stable during the whole process of testing. They may change at any time moment causing the background processes to experience change, which is known as change‐point. Keeping all these critical testing environment factors under consideration, a new software reliability growth model has been proposed, which is derived from an non homogenous Poisson process (NHPP)based unified scheme for multi‐release two‐stage fault detection/observation and correction/removal software reliability models. The developed model is numerically illustrated on tandem data set for four releases.

A software reliability growth model for imperfect debugging

Software reliability is one of the most important characteristics of software quality. Its measurement and management technologies during the software life-cycle are essential to produce and maintain quality/reliable software systems. In this paper, we discuss software reliability modeling and its applications. As to software reliability modeling, hazard rate and NHPP models are investigated particularly for quantitative software reliability assessment. Further, imperfect debugging and software availability models are also discussed with reference to incorporating practical factors of dynamic software behavior.

It is one of the important issues in modern software development to make a highly reliable software system by efficient and economical testing. As one of these solutions, many companies have taken in testing-progress management and quality/reliability assessment based on software reliability growth models. Consequently, many software reliability growth models reflecting various development factors and operational environment of the software system have been proposed. Most of these models have assumed the software system as one domain. That is, the models have assumed that the set of the testing-paths influenced by execution of test-cases is extended with testing-progress and it expands to whole software system finally. However, a software system has the structure that the main-component calls plural sub-components ordinarily. Therefore, unless the main-component is tested, the sub-component called from it is not tested. In this chapter, we propose software reliability growth models based on such components' characteristics. Especially, these models reflecting the different testing-environment for composed components are formulated by nonhomogeneous Poisson processes. Furthermore, using fault-detection data observed in actual development projects, we show numerical examples of software reliability assessment and results of goodness-of-fit comparisons with existing software reliability growth models.

Software reliability is one of the most important characteristics of software quality. Its measurement and management technologies during the software life-cycle are essential to produce and maintain quality/reliable software systems. In this chapter, we discuss software reliability modeling and its applications. As to software reliability modeling, hazard rate and NHPP models are investigated particularly for quantitative software reliability assessment. Further, imperfect debugging and software availability models are also discussed with reference to incorporating practical factors of dynamic software behavior. And three software management problems are discussed as an application technology of software reliability models: the optimal software release problem, statistical testing-progress control, and the optimal testing-effort allocation problem.

The safety-critical software of Reactor Protection System (RPS) plays a significant role for the safe operation of the nuclear power plant (NPP). However, it also brings challenges both to the reliability analysis of the RPS and to the Probabilistic Risk Assessment of the NPPs. The reliability analysis of safety-critical software is also expected by the nuclear regulation agencies and the software development groups for test evaluation and optimization. The detected faults during the software test process are regarded to have close connection with the software reliability and there have been hundreds of test-based software reliability models. Due to the particularity of its function, the safety-critical software of an NPP is especially sensitive to the faults with higher severity levels which should be paid special attention to. Severity levels of faults are commonly taken into account during software reliability modelling. In this paper, a novel software reliability growth model considering actual severity data was built based on a non-homogeneous Poisson process. Ratio of fatal faults with highest severity level to all accumulated faults was modelled with logistic curve. Then the mean value functions of both fatal and general faults were derived. A belief factor was employed to describe the trend of severity classification. In the end, the test data of a practical project was used to verify this model. The analysis results showed that this model had better prediction effect than Goel-Okumoto model and Inflection S-shaped model even when limited test data were collected. This novel model can be used as a helpful tool to evaluate both the reliability and the release time of the safety-critical software of an NPP.

Reliability of composite software by different forms of uncertainty

Software reliability growth models (SRGMs) have been developed to estimate software reliability measures such as the number of remaining faults, software failure rate, and software reliability. Issues such as imperfect debugging and the learning phenomenon of developers have been considered in these models. However, most SRGMs assume that faults detected during tests will eventually be removed. Consideration of fault removal efficiency in the existing models is limited. In practice, fault removal efficiency is usually imperfect. This paper aims to incorporate fault removal efficiency into software reliability assessment. Fault removal efficiency is a useful metric in software development practice and it helps developers to evaluate the debugging effectiveness and estimate the additional workload. In this paper, imperfect debugging is considered in the sense that new faults can be introduced into the software during debugging and the detected faults may not be removed completely. A model is proposed to integrate fault removal efficiency, failure rate, and fault introduction rate into software reliability assessment. In addition to traditional reliability measures, the proposed model can provide some useful metrics to help the development team make better decisions. Software testing data collected from real applications are utilized to illustrate the proposed model for both the descriptive and predictive power. The expected number of residual faults and software failure rate are also presented.

In reality, the fault detection (or correction) phenomenon and software reliability estimation in the operational phase are different from that in the testing phase. The fault removal continues at a slower rate during the operational phase. In this paper, we will investigate some techniques for software reliability prediction and measurement in the operational phase. We first review how some software reliability growth models (SRGMs) based on non-homogeneous Poisson processes (NHPPs) can be readily derived based on a unified theory for NHPP models. Under this general framework, we can not only verify some conventional SRGMs but also derive some new SRGMs that can be used for software reliability measurement in the operational phase. That is, based on the unified theory, we can incorporate the concept of multiple changepoints into software reliability modeling. We can formularize and simulate the fault detection process in the operational phase. Some numerical illustrations based on real software failure data are also presented. The experimental results show that the proposed models can easily reflect the possible changes of fault detection process and offer quantitative analysis on software failure behavior in field operation.

The non-homogeneous Poisson process (NHPP) model is a very important class of software reliability models and is widely used in software reliability engineering. NHPPs are characterized by their intensity functions. In the literature it is usually assumed that the functional forms of the intensity functions are known and only some parameters in intensity functions are unknown. The parametric statistical methods can then be applied to estimate or to test the unknown reliability models. However, in realistic situations it is often the case that the functional form of the failure intensity is not very well known or is completely unknown. In this case we have to use functional (non-parametric) estimation methods. The non-parametric techniques do not require any preliminary assumption on the software models and then can reduce the parameter modeling bias. The existing non-parametric methods in the statistical methods are usually not applicable to software reliability data. In this paper we construct some non-parametric methods to estimate the failure intensity function of the NHPP model, taking the particularities of the software failure data into consideration.

Non-homogeneous Poisson process (NHPP) model with typical reliability growth patterns is an important technology about evaluating software reliability. Two parameters which affect software reliability are original failures number and failure-detected rate. The paper firstly defines the software failure distributed and discusses a few kinds' software reliability models. Non-homogeneous Poisson process model is presented. The models with different parameters are discussed and analyzed. The model's restrict condition and some of parameters, original failures number and failure-detected ratio, are extrapolated. Assessment methodology about key parameters is given.