GeCodP: Test Case Generation Under the Prism of Distributed Applications

Services are highly reusable, flexible and loosely coupled, which makes the evolution and the maintenance of composite services more complex. Evolution of BPEL composite service covers changes of processes, bindings and interfaces. In this paper, an approach is proposed to select and generate test cases during the evolution of BPEL composite service. The approach identifies the changes by using control-flow analysis technique and comparing the paths in new composite service version and the old one using extensible BPEL flow graph (or XBFG). Message flow is appended to the control flow so that XBFG can describe the behavior of composite service integrally. The binding and predicate constraint information added in XBFG elements can be used in path selection and test case generation. Theory analysis and case study both show that the approach is effective, and test cases coverage rate is high for the changes of processes, bindings and interfaces.

SummaryThis article presents the techniques and results of a novel model‐based test case generation approach that automatically derives test cases from UML state machines. The main contribution of this article is the fully automated fault‐based test case generation technique together with two empirical case studies derived from industrial use cases. Also, an in‐depth evaluation of different fault‐based test case generation strategies on each of the case studies is given and a comparison with plain random testing is conducted. The test case generation methodology supports a wide range of UML constructs and is grounded on the formal semantics of Back's action systems and the well‐known input–output conformance relation. Mutation operators are employed on the level of the specification to insert faults and generate test cases that will reveal the faults inserted. The effectiveness of this approach is shown and it is discussed how to gain a more expressive test suite by combining cheap but undirected random test case generation with the more expensive but directed mutation‐based technique. Finally, an extensive and critical discussion of the lessons learnt is given as well as a future outlook on the general usefulness and practicability of mutation‐based test case generation. Copyright © 2014 John Wiley & Sons, Ltd.

Software testing is the key to ensuring software quality. Test case generation and software fault localization are two important research objects of testing. Spectrum-based fault localization (SBFL) method is a mainstream dynamic fault localization method, which inputs test cases into the program for execution and collects statement coverage information and execution results. By analysing this information, a fault report can be generated. The performance of SBFL is affected by test case quality. Therefore, this paper proposes a test case generation method based on improved genetic algorithm to assist SBFL. This method uses a small number of initial test cases to execute the program, then uses the XGBoost model to pre-process the spectrum information and generates a list of potential fault statements to guide the iterative process of the genetic algorithm. The experimental results show that the test cases generated using the improved genetic algorithm have certain advantages over the test cases generated by general genetic algorithm and the baseline test cases.

Test cases are essential parts to perform software testing as they play very significant role defining the test steps to work. Test cases can be generated either manually or automatically. Creating test cases manually has several disadvantages, such as higher probability in producing inaccurate results. Furthermore, generating test cases manually is ineffective since manual methods will take longer and require high human efforts. Therefore, generating test cases automatically is more preferable to limit the previous problems. Automated testing can also provide more accurate results than manual testing because it is less prone to human errors encouraging software developers to generate test cases automatically. Test cases can be generated from various sources, including UML diagrams. In this paper, we propose a method to generate test cases automatically by utilising depth-first search algorithms. The UML diagram used in this study is an activity diagram. This paper also provides a method to parse activity diagram that is created using the visual paradigm application that has been converted into an XML format. Our method is able to parse and convert the XML format into graph or tree form. Using our proposed method, a test case search will be carried out using the depth-first search on the tree form to generate test cases. Lastly, we also evaluate our method by comparing the generated test cases with cyclomatic complexity, the method will be performed manually and automatically to validate the results of many scenarios. Our result showed that our method could generate the same scenario (100% coverage) as the manual validation method.

Maximum coverage with minimum testing time is the main objective of a test case generation activity which leads to a multi-objective problem. Search-Based Testing (SBT) technique is a demanding research area for test case generation. Researchers have applied various metaheuristic (searching) algorithms to generate efficient and effective test cases in many research works. Out of these existing search-based algorithms, Genetic Algorithm (GA) and Particle Swarm Optimization (PSO) are the most widely used algorithms in automatic test case generation. In this paper, a test case generation approach is proposed using Cuckoo Search (CS) algorithm. CS has a controlling feature, Levy flights, which makes it more efficient in searching the best candidate solution. It helps to generate efficient test cases in terms of code coverage and execution time. In our proposed method, test cases are generated based on path coverage criteria. Fitness of a test case is evaluated using branch distance and approximation level combined functions. The result is compared with PSO and with its variant Adaptive PSO (APSO). The experimental result shows that both the algorithms give nearly equal to the same result. Though the results are nearly equal, the implementation of CS is simple as it requires only one parameter to be tuned.

Extended finite state machine (EFSM) is among the most popular models for model-based testing. However, automated test case generation on EFSM models is still a challenging task since an EFSM model may contains infeasible paths. This paper proposed a novel approach (ATGEM) to generate test case and construct oracle information from EFSM automatically. To address the infeasible problem, a metric based on data flow analysis is presented to predict the infeasible probability so as to bypass the infeasible paths as far as possible and improve the test case generation efficiency. Afterwards, an executable EFSM model is developed to obtain runtime feedback information as a fitness function in order to generate test data and construct oracle information automatically. This approach, which can generate various types data and has a wide range of applications, combines static analysis and dynamic analysis aims to find a preferable feasible path subset to generate test cases and meet adequacy coverage criteria. The experimental results on several EFSM models show that test case generation method and path feasibility metric have good effectiveness. Utilizing path feasibility metric can speed up the process of test case generation greatly, and ATGEM is more efficient than existing method.

This paper presents work on generation of specification-driven test cases based on quasirandom (low-discrepancy) sequences instead of pseudorandom numbers. This approach is novel in software testing. This enhanced uniformity of quasirandom sequences leads to faster generation of test cases covering all possibilities. We demonstrate by examples that quasirandom sequences can be a viable alternative to pseudorandom numbers in generating test cases. In this paper, we present a method that can generate test cases from a decision table specification more effectively via quasirandom numbers. Analysis of a simple problem in this paper shows that quasirandom sequences achieve better data than pseudorandom numbers, and have the potential to converge faster and so reduce the computational burden. The use of different quasirandom sequences for generating test cases is presented in this paper.

Generating test cases from the formal specification description VDM++, which is a method for disambiguating specifications, is time-consuming and labor intensive.Therefore, our laboratory has developed BWDM, a tool that automatically generates test cases from VDM++ specifications.However, existing BWDM has problems that it only supports integer types and cannot generate test cases for operation and function definitions including recursive structure.Therefore, in order to improve the usefulness of BWDM, this paper extends BWDM to solve the above problems.Consequently, it has confirmed that the use of extended BWDM can reduce the test case generation time compared to manual test case generation.

Testing of product is perform to discover or detect the errors and defects in the developed system. But testing is usually time consuming especially when complex projects are canvass. Testing of a product lead off with generation of test cases. The Test case generation are based on three parts coding, design and specification. The Specification based testing deals with generation of test cases from the functional requirements. The proposed system deals with automatic generation of test cases from functional requirement using Natural Language Processing (NLP). The proposed system constructs test cases based on keywords in context from functional requirement of Software Requirement Specification document. The proposed system is beneficial as it can automatically analyze the functional requirement from Software Requirement Specification in order to extract test cases for testing. The goal of the proposed system is to reduce effort and time consumed by software tester to test the product.

Dependence relations have been used in slicing of programs in order to remove statements that do not influence certain criteria of interest. More recently, slicing has also been applied at the specification level in order to obtain a reduced model pertinent to the selected criteria. Such models have been used for different verification and validation activities. In this article we present an approach that uses control, data and communication dependences in order to enhance test purposes with refuse transitions. A test purpose represents an abstraction of the original model describing a scenario of interest, which should be tested. The refuse transitions are used during the test case generation process in order to limit the state space being searched. As automating test case generation activities is of great importance the generation of the test purposes and of the test cases consequently is fully automatic. We have evaluated the proposed approach on three models from industry and several others from literature. The obtained results indicate an improvement regarding computation time compared to test case generation using unmodified test purposes.

The effectiveness and precision of network-based intrusion detection signatures can be evaluated either by direct analysis of the signatures (if they are available) or by using black-box testing (if the system is closed-source). Recently, several techniques have been proposed to generate test cases by automatically deriving variations (or mutations) of attacks. Even though these techniques have been useful in identifying "blindspots" in the signatures of closed-source, network-based intrusion detection systems, the generation of test cases is performed in a random, un- guided fashion. The reason is that there is no information available about the signatures to be tested. As a result, identifying a test case that is able to evade detection is difficult. In this paper, we propose a novel approach to drive the generation of test cases by using the information gathered by analyzing the dynamic behavior of the intrusion detection system. Our approach applies dynamic dataflow analysis techniques to the intrusion detection system to identify which parts of a network stream are used to detect an attack and how these parts are matched by a signature. The result of our analysis is a set of constraints that is used to guide the black-box testing process, so that the mutations are applied to only those parts of the attack that are relevant for detection. By doing this, we are able to perform a more focused generation of the test cases and improve the process of identifying an attack variation that evades detection.

Based on our experience of collaborating with industry, we observed that test case generation usually relies on test case specifications (TCSs), commonly written in natural language, specifying test cases of a System Under Test at a high level of abstraction. In practice, TCSs are commonly used by test engineers as reference documents to perform these activities: 1) Manually executing test cases in TCSs; 2) Manually coding test cases in a test scripting language for automated test case execution. In the latter case, the gap between TCSs and executable test cases has to be filled by test engineers, requiring a significant amount of coding effort and domain knowledge. Motivated by the above observations from the industry, we first propose, in this paper, a TCS language, named as Restricted Test Case Modeling (RTCM), which is based on natural language and composed of an easy-to-use template, a set of restriction rules and keywords. Second, we propose a test case generation tool (aToucan4Test), which takes TCSs in RTCM as input and generates either manual test cases or automatically executable test cases, based on various coverage criteria defined on RTCM. To assess the applicability of RTCM, we manually modeled two industrial case studies and examined 30 automatically generated TCSs. To evaluate aToucan4Test, we modeled three subsystems of a Video Conferencing System developed by Cisco Systems, Norway and automatically generated executable test cases. These test cases were successfully executed on two commercial software versions. In the paper, we also discuss our experience of applying RTCM and aToucan4Test in an industrial context and compare our approach with other model-based testing methodologies.

As we know, software testing is an important part of software development lifecycle. More than 50% of the whole system development work and total cost were spend on the software testing. And it’s estimated that almost 60% of the total test time and cost were spent on the design of test cases. In recent years, the automated testing gradually replaces the traditional manual testing to become an important branch of software testing. In automated testing, the most important thing is to design and generate valid test case automatically. In this paper, In order to reduce the workload of testers on the test case generation, improve the test efficiency and makes test experiences be passed on, we design a common description method for the test case based on ontology; propose a semantic similarity measure method to retrieve the usable test cases from test case library based on WordNet; establish the relationship between the test case and test requirement though rules to modify the retrieved test case; generate the final test case sequence. At last, the practical applicability of the approach is evaluated through an experiment. Keywords-software testing; ontology; test case generation; semantic similarity measure

The amount of test cases in real life software testing is huge. There are many studies, such as classification tree and class vector, on generating all the test cases at a time. The practicability of these studies is low as the testers are difficult to manage that huge amount of test cases. When testing a subset of functions of a system or testing the part of systems that have been modified, it would be more practical and convenience for testers if only all the concerned test cases are generated on demand. In order to generate test cases on demand, first we need a modeling language which is capable to model the input. The modeling language should be elegant and convenient such that testers can focus in the testing. This modeling language should support generating test cases according to some criteria as well. Furthermore, the model language should be implementable for computer aided test cases generation. We found hypergraph is a good candidate as the modeling language. In this paper, we report our study on using hypergraph as the modeling language for generating test cases on demand.

In the past decades, automated test case generation for detecting software vulnerabilities using the search-based algorithm has been a crucial research area for security researchers. Several proposed techniques partially achieve the automatic test case generation of detecting security issues that target specific programming languages or API types. This review paper performs a comparative analysis of research papers published during 2010–21 specific to the search-based techniques used for automatic test case generation. We have chosen five criteria for evaluating the proposed techniques. These are target vulnerability, search-based algorithms used, test case generation source, test case generation method, and target language. We focused primarily on the techniques detecting the four most dangerous vulnerabilities found in the modern distributed system, namely Denial of Service (e.g., program crash), SQL injection, Cross-Site Scripting, and XML injection. We also incorporated the fitness function comparison table for each method, including the merits and limitations. This work will assist security researchers in finding new directions in the field of search-based software security testing.KeywordsSecurity testingSearch-based software security testing (SBSST)SBSEProgram crashDoSXMLISQLIXSS