Usage modeling through sequence enumeration for automated statistical testing of a GUI application

The paper concerns automatic generation of usage models from real-time software UML models. Firstly, we define the reasonably constrained real-time software UML artifacts, which include use case diagrams, timed sequence diagrams and the execution probability of each sequence diagram in its associated use case. Secondly, the paper presents a method that derives the software usage model from the constrained UML artifacts. The method elicits the messages associated with the objects under testing and their occurrence probabilities to generate the usage model of each use case. Timing constraints in sequence diagrams are considered during usage model generation. Then the usage models of use cases are integrated into the software usage model by utilizing the execution sequence relations between use cases. The usage models can be used to generate real-time software statistical test cases and facilitate real-time software statistical testing.

When a population is too large for study, as is the case for all possible uses of a software system, a statistically correct sample must be drawn as a basis for inferences about the population. In statistical testing of software based on a Markov chain usage model, the rich body of analytical results available for Markov chains provides numerous insights that can be used in test planning. Further, the connection between Markov chains and operations research techniques permits a Markov usage model to be expressed as a system of constraints, with mathematical programming used to generate the optimal model for a particular objective function. Since a software usage model is based on the specification, all analyses may be performed early in the development cycle and used as a quantitative basis for management decisions. These techniques have been reduced to engineering practice and used in large projects by IBM, Ericsson, all branches of the US military, and others. In this paper, statistical experiments, Markov models, and optimization techniques are shown to provide a sound theoretical and practical basis far quantifying the reliability of software.

Statistical testing based on a Markov chain usage model, as a rigorous testing method, has been around for more than two decades. Through the comprehensive application of statistical science to the testing of software, it provides audit trails of evidence to support correctness arguments for a software-intensive system as well as a decision that the system is of requisite quality for its intended use. This paper reports a real-world case study in which we applied standard statistical testing practices to the phone flag copy testing problem at the site of our industrial collaborator, and presents our solution from problem formalization, usage modeling and model analysis to test case generation and analysis. Our results helped evaluate the coverage of a heuristically generated test suite, and shed light on what other test cases to craft/generate to improve test coverage and adequacy, as well as reliability estimates both at the arc (usage event) level and at the path (system) level.

This position paper describes an approach to create a framework for modeling affect in decision making for agile processes, and a procedure to test its use by applying it to the introduction of Multi Criteria Decision Methods, and in particular the Best Worst Method, into agile development. We believe that affect changes development in a significant way especially for agile development, which requires close collaboration. We postulate that the structuring of a decision will engage a larger audience eliciting full participation by actors like the customer who are not necessarily accustomed to a development environment. Further, we believe that the structuring of the decision process and the involvement of a larger pool of actors will reduce negative affect, thus enabling a faster, better empowering set of decisions that ultimately will result in higher quality software products and a lower development time.

Software product lines (SPLs) are families of software systems sharing common assets and exhibiting variabilities specific to each product member of the family. Commonalities and variabilities are often represented as features organized in a feature model. Due to combinatorial explosion of the number of products induced by possible features combinations, exhaustive testing of SPLs is intractable. Therefore, sampling and prioritization techniques have been proposed to generate sorted lists of products based on coverage criteria or weights assigned to features. Solely based on the feature model, these techniques do not take into account behavioural usage of such products as a source of prioritization. In this paper, we assess the feasibility of integrating usage models into the testing process to derive statistical testing approaches for SPLs. Usage models are given as Markov chains, enabling prioritization of probable/rare behaviours. We used featured transition systems, compactly modelling variability and behaviour for SPLs, to determine which products are realizing prioritized behaviours. Statistical prioritization can achieve a significant reduction in the state space, and modelling efforts can be rewarded by better automation. In particular, we used MaTeLo, a statistical test cases generation suite developed at ALL4TEC. We assess feasibility criteria on two systems: Claroline, a configurable course management system, and Sferion™, an embedded system providing helicopter landing assistance.

Importance sampling (IS) is a change-of-measure technique for speeding up the simulation of rare events in stochastic systems. In this paper, we presented an approach uses Importance Sampling technique for efficient estimation of software reliability via Markov software usage models in statistical testing. By suitable changes of the probabilities of state transitions during test, an iterative method based on the Ali-Silvey distance is proposed for this choice, and an unbiased reliability estimator with zero variance is obtained. A learning algorithm for the computation of optimal transition probabilities of the Markov chain usage model is also presented and experimental results of this algorithm are reported.

Software stakeholders, including developers, managers, and end users, require high quality software products. Several works have aimed to identify software quality, but the quality of software products is often not comprehensively, specifically, or effectively defined because previous approaches have focused on certain quality aspects. Moreover, the evaluation results of quality metrics often depend on software stakeholders so that it is often hard to compare quality evaluation results across software products. ISO/IEC has tried to define evaluation methods for the quality of software products and provide common standards, called the SQuaRE (Systems and software Quality Requirements and Evaluation) series including ISO/IEC 25022:2016 and ISO/IEC 25023:2016. However, the SQuaRE series include ambiguous metrics so that it is not always easy to apply the series to products and compare results. In this paper, we propose a SQuaRE-based software quality evaluation framework, which successfully concretized many product metrics and quality in use metrics originally defined in the SQuaRE series1. Through a case study targeting a commercial software product, we confirmed that our framework is concretely applicable to the software package/service product.

Nowadays, software industry is still facing many problems in controlling and evaluating software product quality. One of the primary reasons for this is that except for the lack of an objective software product quality assessment model, software organizations do not have a well-defined mechanism for measuring the quality attributes and further evaluating the level of software product quality. This paper proposes a process model for evaluating the level of software product quality, which is based on the International Standard ISO/IEC 14598 - Software Product Evaluation. The proposed process model can generate a tailored software product quality evaluation model based on the types of information systems. Accordingly, the required software measures are collected and further analyzed for providing feedback to improve the software product quality. It can help software development organizations establish their own evaluation models of the software product quality level and thus serve as an agreement of the requirement of software product quality.

CenUMs – concurrency enhanced usage models for statistical testing of complex systems with concurrent streams of use

The software fault classification is very crucial in the development of reliable and high-quality software products. The fault classification allows determining and concentrating on fault software modules for early prediction of fault in time. As a result, it saves the time and money of the industry. Generally, various metrics are generated to represent the fault. But, selecting the dominant metrics from the available set is a challenge. Therefore, in this paper, a sequential forward search (SFS) with extreme learning machine (ELM) approach has used for fault classification. The number of features available in the metrics are selected to represent the fault using SFS and operated on ELM to verify the performance of software fault classification. Also, various activation functions of ELM have tested for the proposed work to identify the best model. The experimental result demonstrates that ELM with radial basis function achieves the good results compared to other activation function. Also, the proposed method has shown good results in comparison to support vector machine.

Cleanroom software engineering requires statistical testing by an independent agent for the purpose of certifying software quality. Statistical software testing is a formal process that involves sampling from the intended usage environment and the precise measurement of properties of random variables inherent in such a statistical experiment. The first step in conducting a statistical test is to determine the usage distribution for the software in its intended environment. This distribution is the basis for a random generator of test sequences for the software. In order to achieve this distribution, a usage analysis is performed using the software specification and any available usage information. The usage analysis consists of a top down, structural investigation of the specification document that establishes a set of usage states and defines an ordering on this set. The paper describes in detail the usage analysis and inference procedure including various computations on the ensuing Markov chains. Stopping criteria are developed and a discrete software reliability model is presented.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Abstract: This study report offers a thorough analysis of current studies on DevOps Agile techniques. The study summarizes and dissects the major discoveries made in peer-reviewed literature, conference proceedings, and other academic publications. The article examines the most recent DevOps Agile practice trends and advances, as well as how they affect software development, testing, deployment, and operations. The analysis emphasizes the advantages of applying DevOps Agile approaches, including enhanced collaboration, faster time to market, and higher-quality software products. The paper also examines the challenges and potential limitations of DevOps Agile adoption, such as cultural resistance, lack of skills, and integration issues. Finally, the study concludes with recommendations for practical implications for software development teams and organizations seeking to implement DevOps Agile methodologies. The review provides valuable insights and guidance for researchers, practitioners, and decision-makers in the field of software engineering and information technology.

Software Product Lines (SPL) are inherently difficult to test due to the\ncombinatorial explosion of the number of products to consider. To reduce the\nnumber of products to test, sampling techniques such as combinatorial\ninteraction testing have been proposed. They usually start from a feature model\nand apply a coverage criterion (e.g. pairwise feature interaction or\ndissimilarity) to generate tractable, fault-finding, lists of configurations to\nbe tested. Prioritization can also be used to sort/generate such lists,\noptimizing coverage criteria or weights assigned to features. However, current\nsampling/prioritization techniques barely take product behavior into account.\nWe explore how ideas of statistical testing, based on a usage model (a Markov\nchain), can be used to extract configurations of interest according to the\nlikelihood of their executions. These executions are gathered in featured\ntransition systems, compact representation of SPL behavior. We discuss possible\nscenarios and give a prioritization procedure illustrated on an example.\n

Interactions between environmental sustainability goals and software product quality: A mapping study

The next generation of consumer goods, including computers, will be much more sophisticated in order to cope with a less technologically literate user base. A user model is an essential component for “user friendliness”, enabling the behavior of a system to be tailored to the needs of a particular user. Simple user profiles already personalise many software products and consumer goods such as digital TV recorders and mobile phones. A user model should be easy to initialise, and it must adapt in the light of interaction with the user. In many cases, a large amount of training data is needed to generate a user model, and adaptation is equivalent to completely retraining the system. This paper briefly outlines the user modelling problem and work done at BTexact on an Intelligent Personal Assistant (IPA) which incorporates a user profile. We go on to describe FILUM, a more flexible method of user modelling, and show its application to the Telephone Assistant and Email Assistant components of the IPA, with tests to illustrate its usefulness. An experimental testbed based on the iterated prisoner’s dilemma, which allows the generation of unlimited data for learning or testing, is also proposed.