Improving Test Case Generation from UML Statecharts by Using Control, Data and Communication Dependencies

Extended finite state machine (EFSM) is among the most popular models for model-based testing. However, automated test case generation on EFSM models is still a challenging task since an EFSM model may contains infeasible paths. This paper proposed a novel approach (ATGEM) to generate test case and construct oracle information from EFSM automatically. To address the infeasible problem, a metric based on data flow analysis is presented to predict the infeasible probability so as to bypass the infeasible paths as far as possible and improve the test case generation efficiency. Afterwards, an executable EFSM model is developed to obtain runtime feedback information as a fitness function in order to generate test data and construct oracle information automatically. This approach, which can generate various types data and has a wide range of applications, combines static analysis and dynamic analysis aims to find a preferable feasible path subset to generate test cases and meet adequacy coverage criteria. The experimental results on several EFSM models show that test case generation method and path feasibility metric have good effectiveness. Utilizing path feasibility metric can speed up the process of test case generation greatly, and ATGEM is more efficient than existing method.

Software systems are not considered complete unless properly tested and verified. In existing literature, a growing interest on establishment of automated testing techniques has been observed. However, tedious manual process of test case generation largely depends upon domain knowledge and formalized representation of user requirements. The advent of semantic web engineering has led the foundation for developing ontologies as a mean to express information and knowledge semantics regarding particular domain efficiently. In software testing, ontologies can be significantly helpful to automate testing phase as they encode domain knowledge in machine interpretable format. We have proposed automatic test case generation framework that involves ontology-based requirement specification and learning based methods for conducting black box testing. Our approach integrates knowledge-based system (ontology) with learning-based testing algorithm to automate: generation of test cases, test execution and test verdict construction. Proposed framework includes, requirement ontology to formalize requirement specification, Dialogue Manager that enables selection of available test cases and Learning Based Testing to generate counter examples of test cases through system learning. The contribution of this paper is to enable 1) requirement elicitation and specification using ontologies 2) test data selection from existing ontologies and 3) automatic test case generation from existing test cases. To represent the applicability of this research, ontology for requirement elicitation and specification is developed. Framework proposed in this research paper is an effort to provide software testing tools to save time, cost and efforts during test design phase.

SummaryThis article presents the techniques and results of a novel model‐based test case generation approach that automatically derives test cases from UML state machines. The main contribution of this article is the fully automated fault‐based test case generation technique together with two empirical case studies derived from industrial use cases. Also, an in‐depth evaluation of different fault‐based test case generation strategies on each of the case studies is given and a comparison with plain random testing is conducted. The test case generation methodology supports a wide range of UML constructs and is grounded on the formal semantics of Back's action systems and the well‐known input–output conformance relation. Mutation operators are employed on the level of the specification to insert faults and generate test cases that will reveal the faults inserted. The effectiveness of this approach is shown and it is discussed how to gain a more expressive test suite by combining cheap but undirected random test case generation with the more expensive but directed mutation‐based technique. Finally, an extensive and critical discussion of the lessons learnt is given as well as a future outlook on the general usefulness and practicability of mutation‐based test case generation. Copyright © 2014 John Wiley & Sons, Ltd.

With the widespread use of smart contracts, security incidents caused by improper programming have drawn increasing attention. Data flow correctness is a fundamental and vital requirement for smart contracts. Although test cases generated by existing approaches for data flow testing of smart contracts can achieve certain coverage, not all def-use (definitionuse) pairs are covered, which may result in some errors not being detected. To further improve the adequacy and efficiency of testing, we present an approach named TCG-Re (T est C ase G eneration Combined with Data Flow R eduction) to generate test cases for data flow testing of smart contracts. Data flow reduction is performed to reduce redundant def-use pairs and obtain the final test requirements in a smart contract. The test case generation process is optimized to improve the coverage of the generated test cases for the def-use pairs. We collect 30 smart contracts of different scales to perform experimental evaluation. The result shows that, compared with other existing approaches, TCG-Re can not only achieve full coverage for most contracts, but also be more efficient. In addition, TCG-Re is also effective for discovering several types of errors in smart contracts.

In software engineering, system modeling is the process of formulating a representation of a real system in an abstract way to understand its behavior. Software testing encourages reusing these models for testing purpose. This expedites the process of test case generation. UML structural and behavioral specification diagrams have been used by testing researchers for generation of test scenarios and test data. The aim of this survey is to improve the understanding of UML based testing techniques. We have focused on test case generation from the behavioral specification diagrams, namely sequence, state chart and activity diagrams. We classify the various research approaches that are based on formal specifications, graph theoretic, heuristic testing, and direct UML specification processing. We discuss the issues of test coverage associated with these approaches.

Test case generation, selection and coverage from natural language

UML activity and state chart diagrams are the most frequently UML models used for testing a system based on its specifications. Testers may choose different UML diagrams based on comprehensibility criteria. Previous studies, however, highlighted the lack of experts’ evaluation of comprehensibility of activity and state chart diagrams in test case generation. Thus, the main objective of this study is to evaluate the comprehensibility of the UML activity and state chart diagrams based on perspectives of experts. This study generated manual test cases from activity and state chart diagrams and used comprehensibility’s evaluation criteria during interviews with several software testing experts. The results of this study revealed that activity diagrams are more comprehensible than state chart diagrams in test case generation by considering comprehension variables (perceived difficulty and subjective confidence). The results of this study aim to unify the practical efforts of testers to choose MBT diagrams in generating test cases.

Services are highly reusable, flexible and loosely coupled, which makes the evolution and the maintenance of composite services more complex. Evolution of BPEL composite service covers changes of processes, bindings and interfaces. In this paper, an approach is proposed to select and generate test cases during the evolution of BPEL composite service. The approach identifies the changes by using control-flow analysis technique and comparing the paths in new composite service version and the old one using extensible BPEL flow graph (or XBFG). Message flow is appended to the control flow so that XBFG can describe the behavior of composite service integrally. The binding and predicate constraint information added in XBFG elements can be used in path selection and test case generation. Theory analysis and case study both show that the approach is effective, and test cases coverage rate is high for the changes of processes, bindings and interfaces.

Fault-based testing is a technique where testers anticipate errors in a system under test in order to assess or generate test cases. The idea is to have enough test cases capable of detecting these anticipated errors. This paper presents a theory and technique for generating fault-based test cases for concurrent systems. The novel idea is to generate test purposes from faults that have been injected into a model of the system under test. Such test purposes form a specification of a more detailed test case that can detect the injected fault. The theory is based on the notion of refinement. The technique is automated using the TGV test case generator and an equivalence checker of the CADP tools. A case study of testing web servers demonstrates the practicability of the approach.KeywordsModel CheckerMutation TestingTesting TheoryTest PurposeLabel Transition SystemThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Test cases are essential parts to perform software testing as they play very significant role defining the test steps to work. Test cases can be generated either manually or automatically. Creating test cases manually has several disadvantages, such as higher probability in producing inaccurate results. Furthermore, generating test cases manually is ineffective since manual methods will take longer and require high human efforts. Therefore, generating test cases automatically is more preferable to limit the previous problems. Automated testing can also provide more accurate results than manual testing because it is less prone to human errors encouraging software developers to generate test cases automatically. Test cases can be generated from various sources, including UML diagrams. In this paper, we propose a method to generate test cases automatically by utilising depth-first search algorithms. The UML diagram used in this study is an activity diagram. This paper also provides a method to parse activity diagram that is created using the visual paradigm application that has been converted into an XML format. Our method is able to parse and convert the XML format into graph or tree form. Using our proposed method, a test case search will be carried out using the depth-first search on the tree form to generate test cases. Lastly, we also evaluate our method by comparing the generated test cases with cyclomatic complexity, the method will be performed manually and automatically to validate the results of many scenarios. Our result showed that our method could generate the same scenario (100% coverage) as the manual validation method.

Maximum coverage with minimum testing time is the main objective of a test case generation activity which leads to a multi-objective problem. Search-Based Testing (SBT) technique is a demanding research area for test case generation. Researchers have applied various metaheuristic (searching) algorithms to generate efficient and effective test cases in many research works. Out of these existing search-based algorithms, Genetic Algorithm (GA) and Particle Swarm Optimization (PSO) are the most widely used algorithms in automatic test case generation. In this paper, a test case generation approach is proposed using Cuckoo Search (CS) algorithm. CS has a controlling feature, Levy flights, which makes it more efficient in searching the best candidate solution. It helps to generate efficient test cases in terms of code coverage and execution time. In our proposed method, test cases are generated based on path coverage criteria. Fitness of a test case is evaluated using branch distance and approximation level combined functions. The result is compared with PSO and with its variant Adaptive PSO (APSO). The experimental result shows that both the algorithms give nearly equal to the same result. Though the results are nearly equal, the implementation of CS is simple as it requires only one parameter to be tuned.

Software testing is the key to ensuring software quality. Test case generation and software fault localization are two important research objects of testing. Spectrum-based fault localization (SBFL) method is a mainstream dynamic fault localization method, which inputs test cases into the program for execution and collects statement coverage information and execution results. By analysing this information, a fault report can be generated. The performance of SBFL is affected by test case quality. Therefore, this paper proposes a test case generation method based on improved genetic algorithm to assist SBFL. This method uses a small number of initial test cases to execute the program, then uses the XGBoost model to pre-process the spectrum information and generates a list of potential fault statements to guide the iterative process of the genetic algorithm. The experimental results show that the test cases generated using the improved genetic algorithm have certain advantages over the test cases generated by general genetic algorithm and the baseline test cases.

Testing of product is perform to discover or detect the errors and defects in the developed system. But testing is usually time consuming especially when complex projects are canvass. Testing of a product lead off with generation of test cases. The Test case generation are based on three parts coding, design and specification. The Specification based testing deals with generation of test cases from the functional requirements. The proposed system deals with automatic generation of test cases from functional requirement using Natural Language Processing (NLP). The proposed system constructs test cases based on keywords in context from functional requirement of Software Requirement Specification document. The proposed system is beneficial as it can automatically analyze the functional requirement from Software Requirement Specification in order to extract test cases for testing. The goal of the proposed system is to reduce effort and time consumed by software tester to test the product.

The vastly growing complexity of software enabled functionality – especially in the automotive industry where it is usually distributed over a network – requires also a steadily increasing number of test cases to ensure a high product quality. The automatic generation of test cases can ease this situation, but suffers from the well-known state space explosion problem, which is caused by the often infinite number of possible execution paths. The state space can be minimized by rules applied to test case generation leading to an enhanced scalability. A flexible rule adjustment according to the model structure allows a tailored test case generation capable of being fast as well as complete according to a given goal. In this work the implications of such rules on the searching capabilities are studied. The analysis is performed on a symbolic test generation engine based on a network of timed, deterministic UML state machines.

Sequence covering array (SCA) generation is an active research area in recent years. Unlike the sequence-less covering arrays (CA), the order of sequence varies in the test case generation process. This paper reviews the state-of-the-art of the SCA strategies, earlier works reported that finding a minimal size of a test suite is considered as an NP-Hard problem. In addition, most of the existing strategies for SCA generation have a high order of complexity due to the generation of all combinatorial interactions by adopting one-test-at-a-time fashion. Reducing the complexity by adopting one-parameter- at-a-time for SCA generation is a challenging process. In addition, this reduction facilitates the supporting for a higher strength of coverage. Motivated by such challenge, this paper proposes a novel SCA strategy called Dynamic Event Order (DEO), in which the test case generation is done using one-parameter-at-a-time fashion. The details of the DEO are presented with a step-by-step example to demonstrate the behavior and show the correctness of the proposed strategy. In addition, this paper makes a comparison with existing computational strategies. The practical results demonstrate that the proposed DEO strategy outperforms the existing strategies in term of minimal test size in most cases. Moreover, the significance of the DEO increases as the number of sequences increases and/ or the strength of coverage increases. Furthermore, the proposed DEO strategy succeeds to generate SCAs up to t=7. Finally, the DEO strategy succeeds to find new upper bounds for SCA. In fact, the proposed strategy can act as a research vehicle for variants future implementation.