Conformance Testing Based on UML State Machines

Use cases are commonly used to structure and document requirements while UML state machine diagrams often describe the behavior of a system and serve as a basis to automate test case generation in many model-based testing (MBT) tools. Therefore, automated support for the transition from use cases to state machines would provide significant, practical help for testing system requirements. Additionally, traceability could be established through automated transformations, which could then be used for instance to link requirements to design decisions and test cases, and assess the impact of requirements changes. In this paper, we propose an approach to automatically generate state machine diagrams from use cases while establishing traceability links. Our approach is implemented in a tool, which we used to perform three case studies, including an industrial case study. The results show that high quality state machine diagrams can be generated, which can be manually refined at reasonable cost to support MBT. Automatically generated state machines showed to largely conform to the actual system behavior as evaluated by a domain expert.

Testing mobile application refers to the process of ensuring the quality and functionality of the software applications implemented for mobile phones. It is a fundamental step to test mobile applications specially before being released to the market. Load testing is a kind of software testing which is carried out to test the performance of mobile applications under different workloads. However, most of mobile apps are implemented within a bounded resource, budget, and time due to the market pressure. This extremely influences the quality of the mobile applications development industry. The automatic test case generation is the key factor for efficient and cost-effective software verification process. Model-based load testing represents both performance aspects and functional aspects of mobile applications by using modeling languages like unified modeling language (UML). Then, adopting the resulting models to automatically design test cases that will be used for load testing. This paper proposes an automatic model-based test case generation approach for mobile applications load testing. The proposed approach lessens testing time and satisfies fast time-to market constraint of mobile apps. Additionally, the proposed approach validates the mobile applications functionalities as well as the performance requirements.

Requirements-based testing is one of the most commonly used ways to ensure the correctness of software, especially for embedded control software in safety-critical domains such as spacecraft and railway systems. Many industrial standards such as the DO-333 and EN50128 also request rigorous requirements-based software testing. To test embedded control software effectively and efficiently, generating high-quality test cases automatically is extremely important. However, existing methods for generating test cases from requirements require intensive manual efforts and expertise. To address this problem, we proposed an automatic requirements-based software testing method for embedded control software. To obtain automatic test case generation and precise test oracles derivation, requirements specification should be precise and readable for the industrial practitioners. Therefore, we use the light-weight domain-specific formal description language, CASDL (Casco Accurate Specification Description Language) for the industrial practitioners to define software requirements into formal specifications at the first step. Based on the formal specification, we propose an algorithm to automatically generate test inputs that satisfy the MC/DC criteria suggested by typical industrial standards and precise test oracles can be derived by "running" the specification with such test inputs. To this end, we proposed an algorithm for simulating the formal specification to generate the test oracles, i.e., the expected outputs corresponding to the test inputs. To facilitate the application of this method in the industry, we have built a tool that can automatically perform the overall testing process. To validate and evaluate its effectiveness in real industrial projects, we have applied it in testing a real Automatic Train Protection (ATP) system provided by our industrial partner, the Casco Signal Co., Ltd (one of the largest railway control system companies in China). In the case study on ATP requirements, our approach generated test cases for 129 requirement items following MC/DC criteria and caught 40 inconsistencies between Casco's requirements and implementation.

Test case generation, selection and coverage from natural language

SummaryThis article presents the techniques and results of a novel model‐based test case generation approach that automatically derives test cases from UML state machines. The main contribution of this article is the fully automated fault‐based test case generation technique together with two empirical case studies derived from industrial use cases. Also, an in‐depth evaluation of different fault‐based test case generation strategies on each of the case studies is given and a comparison with plain random testing is conducted. The test case generation methodology supports a wide range of UML constructs and is grounded on the formal semantics of Back's action systems and the well‐known input–output conformance relation. Mutation operators are employed on the level of the specification to insert faults and generate test cases that will reveal the faults inserted. The effectiveness of this approach is shown and it is discussed how to gain a more expressive test suite by combining cheap but undirected random test case generation with the more expensive but directed mutation‐based technique. Finally, an extensive and critical discussion of the lessons learnt is given as well as a future outlook on the general usefulness and practicability of mutation‐based test case generation. Copyright © 2014 John Wiley & Sons, Ltd.

Software systems are not considered complete unless properly tested and verified. In existing literature, a growing interest on establishment of automated testing techniques has been observed. However, tedious manual process of test case generation largely depends upon domain knowledge and formalized representation of user requirements. The advent of semantic web engineering has led the foundation for developing ontologies as a mean to express information and knowledge semantics regarding particular domain efficiently. In software testing, ontologies can be significantly helpful to automate testing phase as they encode domain knowledge in machine interpretable format. We have proposed automatic test case generation framework that involves ontology-based requirement specification and learning based methods for conducting black box testing. Our approach integrates knowledge-based system (ontology) with learning-based testing algorithm to automate: generation of test cases, test execution and test verdict construction. Proposed framework includes, requirement ontology to formalize requirement specification, Dialogue Manager that enables selection of available test cases and Learning Based Testing to generate counter examples of test cases through system learning. The contribution of this paper is to enable 1) requirement elicitation and specification using ontologies 2) test data selection from existing ontologies and 3) automatic test case generation from existing test cases. To represent the applicability of this research, ontology for requirement elicitation and specification is developed. Framework proposed in this research paper is an effort to provide software testing tools to save time, cost and efforts during test design phase.

UML is widely accepted and practiced in industries for modeling and design of software systems. Software requirements and design are very important parts of software development, which must be validated and verified before implementation. UML State Machine diagram is an important formalism to model the dynamics of the system. In this paper, a comparative study of the test case generation techniques from UML State Machine diagram has been done. This study is an attempt to find out the work that has already been done in this field and various aspects of UML state machines that have been considered for generating test cases. Finally, this paper presents a scope to generate test data for complex state machine involving concurrent states and events.

This paper describes a research result on automatic generation of abstract test cases from INAP protocol in formal specifications by using and applying many existing related algorithms and techniques such as the testing framework, rural Chinese postman tour and UTO (Unique Input Output) sequence concepts. We use the I/OFSM (Input/Output Finite State Machine) generated from SDL specifications and a characterizing sequence, called Unique Input/Output (UIO) sequence, is defined for the I/O FSM. The UIO sequence is combined with the concept of rural Chinese postman tour, without reliable reset capability, to obtain an optimal test sequence. To develop this method, first of all, we establish a framework leading to formal testing methodology which can be used to assess the conformance of an IUT (Implementation Under Test) to the behaviours specified in an I/OFSM which is equivalent to corresponding FDT (Formal Description Technique) specification. By this framework, we present a methodology for automatic test generation from FDT specifications. A prototype of the proposed framework has been built with special attention to real application in order to generate the executable test cases in an automatic way.

A test case generation tool for conformance testing of SDL systems

UML activity and state chart diagrams are the most frequently UML models used for testing a system based on its specifications. Testers may choose different UML diagrams based on comprehensibility criteria. Previous studies, however, highlighted the lack of experts’ evaluation of comprehensibility of activity and state chart diagrams in test case generation. Thus, the main objective of this study is to evaluate the comprehensibility of the UML activity and state chart diagrams based on perspectives of experts. This study generated manual test cases from activity and state chart diagrams and used comprehensibility’s evaluation criteria during interviews with several software testing experts. The results of this study revealed that activity diagrams are more comprehensible than state chart diagrams in test case generation by considering comprehension variables (perceived difficulty and subjective confidence). The results of this study aim to unify the practical efforts of testers to choose MBT diagrams in generating test cases.

In this paper, we present an approach for accelerating model-based robustness testing of state-based software components. We use the Z3 SAT solver to derive executable test cases fully automatically from a state model based on a UML Statechart. The test cases are paths along the Statechart comprising states and transitions. The main advantage of our approach is an accelerated execution of the test cases by refining already reached states as starting points for further test cases using reverse execution. Furthermore, the presented approach is able to check whether a correct target state has been reached or not when executing the path-based test cases using runtime verification and therefore increases the test significance. The runtime verification is done without instrumenting the source code of the component under test.

Maximum coverage with minimum testing time is the main objective of a test case generation activity which leads to a multi-objective problem. Search-Based Testing (SBT) technique is a demanding research area for test case generation. Researchers have applied various metaheuristic (searching) algorithms to generate efficient and effective test cases in many research works. Out of these existing search-based algorithms, Genetic Algorithm (GA) and Particle Swarm Optimization (PSO) are the most widely used algorithms in automatic test case generation. In this paper, a test case generation approach is proposed using Cuckoo Search (CS) algorithm. CS has a controlling feature, Levy flights, which makes it more efficient in searching the best candidate solution. It helps to generate efficient test cases in terms of code coverage and execution time. In our proposed method, test cases are generated based on path coverage criteria. Fitness of a test case is evaluated using branch distance and approximation level combined functions. The result is compared with PSO and with its variant Adaptive PSO (APSO). The experimental result shows that both the algorithms give nearly equal to the same result. Though the results are nearly equal, the implementation of CS is simple as it requires only one parameter to be tuned.

Separation of Concerns (SoC) is an important issue to reduce the complexity of software. Recent advances in programming language research show that Aspect-Oriented Programming (AOP) may be helpful for enhancing the SoC in software systems: AOP provides a means for describing concerns which are normally spread throughout the whole program at one location. The arguments for introducing aspects into programming languages also hold for modeling languages. In particular, modeling state-crosscutting behavior is insufficiently supported by UML state machines. This often leads to model elements addressing the same concern scattered all over the state machine. We present an approach to aspect-oriented state machines, which show considerably better modularity in modeling state-crosscutting behavior than standard UML state machines.

Background & Problem Statement - Software testing is a critical phase in the software development lifecycle (SDLC), ensuring that applications function correctly, meet user requirements, and maintain high- quality standards. Traditional software testing approaches, including manual testing and rule-based automation, often face challenges in scalability, efficiency, and adaptability to dynamic software environments. Traditional testing methods are overwhelmed by complex software systems which slows down defect detection and extends both testing costs and release schedules. Machine Learning (ML) has emerged as a transformative solution, introducing predictive and adaptive capabilities that optimize test case selection, automate defect detection, and enhance overall software quality assurance (QA). This study explores the integration of ML in software testing, addressing the challenges of traditional QA methodologies and demonstrating how AI-driven frameworks improve testing efficiency. Methodology - To investigate the impact of ML in software testing, this research adopts a systematic approach by analyzing ML-driven test automation techniques, including predictive testing, adaptive test execution, and automated test case generation. Research reviews how Google Microsoft Facebook IBM and Deep Code put ML-based quality assurance frameworks into operation. The study leverages supervised learning, reinforcement learning, deep learning, and NLP-based techniques to demonstrate how ML models predict software defects, dynamically adapt test cases, and optimize testing resources. The research tests how ML-based testing models operate within CI/CD pipelines to improve ongoing testing and deployment flow. Analysis & Results - The analysis of ML-driven software testing reveals that predictive analytics improves early defect detection rates. It helps developers spend 37% less time debugging their work. Adaptive testing models, including self-healing test scripts, minimize maintenance costs by 50% and enhance test reliability in agile environments. The integration of NLP-based test case generation increases test coverage. NLP technology enables automatic connection between requirements and test cases at 89% success rate. Additionally, reinforcement learning techniques improve test case selection, reducing redundant test executions by 43%. Our research shows different ML methods work well to lessen incorrect error alerts. ML integration for QA surely increasing defect prediction accuracy and optimizing test execution time. Findings & Contributions - This research contributes to the field of AI-driven software testing by providing a comprehensive framework for ML-based QA methodologies. Our study shows that machine learning helps find more software problems better adapts test cases and lowers testing expenses to solve present software development needs. The study also identifies critical challenges, including data availability, model interpretability, and computational overhead, suggesting future research directions in Explainable AI (XAI), hybrid AI-ML testing models, and AI-driven security testing. As the industry moves toward AI-first software testing, this research paves the way for fully autonomous QA frameworks, enabling intelligent, scalable, and cost- effective software validation techniques. Keywords - Machine Learning, Software Testing, Quality Assurance, Predictive Testing, Adaptive Testing, Test Automation, Defect Prediction, Self- Healing Test Scripts, AI-Driven QA, Reinforcement

Dependence relations have been used in slicing of programs in order to remove statements that do not influence certain criteria of interest. More recently, slicing has also been applied at the specification level in order to obtain a reduced model pertinent to the selected criteria. Such models have been used for different verification and validation activities. In this article we present an approach that uses control, data and communication dependences in order to enhance test purposes with refuse transitions. A test purpose represents an abstraction of the original model describing a scenario of interest, which should be tested. The refuse transitions are used during the test case generation process in order to limit the state space being searched. As automating test case generation activities is of great importance the generation of the test purposes and of the test cases consequently is fully automatic. We have evaluated the proposed approach on three models from industry and several others from literature. The obtained results indicate an improvement regarding computation time compared to test case generation using unmodified test purposes.