Research on Object-oriented Software Testing Cases of Automatic Generation

Any Instrumentation and Control (I&C) system has to be properly tested to minimize the risk of faults in the system. This is even more crucial if the I&C system implements a safety function. The function of the I&C is often designed in a form of Function Block Diagram (FBD). The presented approach describes a new analytic method for automatic generation of test cases for FBD models. The test cases are generated in order to verify that the implementation behaves as intended. The test cases are designed to be used for gray-box testing of the implemented model. The automatic generation of test cases is generally faster and less prone to miss out an unsafe scenario than manual design.

Various software engineering paradigms and real-time projects have proved that software testing is the most critical and highly important phase in the SDLC. In general, software testing takes approximately 40–60% of the total effort and time involved in project development. Generating test cases is the most important process in software testing. There are many techniques involved in the automatic generation of these test cases which aim to find a smaller group of cases that could allow for an adequacy level to be achieved which will hence reduce the effort and cost involved in software testing. In the structural testing of a product, the auto-generation of test cases that are path focused in an efficient manner is a challenging process. These are often considered optimization problems and hence search-based methods such as genetic algorithm (GA) and swarm optimizations have been proposed to handle this issue. The significance of the study is to address the optimization problem of automatic test case generation in search-based software engineering. The proposed methodology aims to close the gap of genetic algorithms acquiring local minimum due to poor diversity. Here, dynamic adjustment of cross-over and mutation rate is achieved by calculating the individual measure of similarity and fitness and searching for the more global optimum. The proposed method is applied and experimented on a benchmark of five industrial projects. The results of the experiments have confirmed the efficiency of generating test cases that have optimum path coverage.

The vastly growing complexity of software enabled functionality – especially in the automotive industry where it is usually distributed over a network – requires also a steadily increasing number of test cases to ensure a high product quality. The automatic generation of test cases can ease this situation, but suffers from the well-known state space explosion problem, which is caused by the often infinite number of possible execution paths. The state space can be minimized by rules applied to test case generation leading to an enhanced scalability. A flexible rule adjustment according to the model structure allows a tailored test case generation capable of being fast as well as complete according to a given goal. In this work the implications of such rules on the searching capabilities are studied. The analysis is performed on a symbolic test generation engine based on a network of timed, deterministic UML state machines.

Testing is the fundamental technique to assess the correctness of software systems, but it is cost-labored to generate test cases. One solution to change the situation is to automatize some parts of the testing process, especially the generation of test cases using formal theory and technology. The research work in the direction shows the good perspective. This paper targets on the automatic generation of test cases based on IOSTS, which is widely used to model reactive systems with data. When selecting test cases based on a set of test purposes specified by IOSTS or temporal logic, in general, the redundancy phenomena are unavoidable in the derived test suite. Hence, some strategies are presented for eliminating the redundancies in order to reduce the cost of implementing testing. More importantly, the strategies are directly applied to test cases in form of IOSTS, such that it can reduce not only the size of test suite, but also the cost of deriving test cases.

Test case generation, selection and coverage from natural language

Maximum coverage with minimum testing time is the main objective of a test case generation activity which leads to a multi-objective problem. Search-Based Testing (SBT) technique is a demanding research area for test case generation. Researchers have applied various metaheuristic (searching) algorithms to generate efficient and effective test cases in many research works. Out of these existing search-based algorithms, Genetic Algorithm (GA) and Particle Swarm Optimization (PSO) are the most widely used algorithms in automatic test case generation. In this paper, a test case generation approach is proposed using Cuckoo Search (CS) algorithm. CS has a controlling feature, Levy flights, which makes it more efficient in searching the best candidate solution. It helps to generate efficient test cases in terms of code coverage and execution time. In our proposed method, test cases are generated based on path coverage criteria. Fitness of a test case is evaluated using branch distance and approximation level combined functions. The result is compared with PSO and with its variant Adaptive PSO (APSO). The experimental result shows that both the algorithms give nearly equal to the same result. Though the results are nearly equal, the implementation of CS is simple as it requires only one parameter to be tuned.

Software systems are not considered complete unless properly tested and verified. In existing literature, a growing interest on establishment of automated testing techniques has been observed. However, tedious manual process of test case generation largely depends upon domain knowledge and formalized representation of user requirements. The advent of semantic web engineering has led the foundation for developing ontologies as a mean to express information and knowledge semantics regarding particular domain efficiently. In software testing, ontologies can be significantly helpful to automate testing phase as they encode domain knowledge in machine interpretable format. We have proposed automatic test case generation framework that involves ontology-based requirement specification and learning based methods for conducting black box testing. Our approach integrates knowledge-based system (ontology) with learning-based testing algorithm to automate: generation of test cases, test execution and test verdict construction. Proposed framework includes, requirement ontology to formalize requirement specification, Dialogue Manager that enables selection of available test cases and Learning Based Testing to generate counter examples of test cases through system learning. The contribution of this paper is to enable 1) requirement elicitation and specification using ontologies 2) test data selection from existing ontologies and 3) automatic test case generation from existing test cases. To represent the applicability of this research, ontology for requirement elicitation and specification is developed. Framework proposed in this research paper is an effort to provide software testing tools to save time, cost and efforts during test design phase.

This work aims to generate White-box Test Cases from a mainframe NATURAL code fragment, using the Control Flow Graph Technique. Basically, it enables a code fragment analysis, generating its control flow represented by a Graph perspective. As a consequence, it provides the automatic generation of White-box Test Cases. Furthermore, this work contributes to attain a lower degree of difficult in the execution of White-box Tests within a Mainframe environment. Also, it brings a significant contribution related to the execution time, inherent to the software testing. At the end, it is added a testing expertise to NATURAL development teams. Usually there is not enough available time to test all possible paths in a algorithm, even in a simple application. Based on it, this work provides automatic generation of test cases for empowerment of team decision, regarding which test cases are more relevant to be performed.

Software testing is the key to ensuring software quality. Test case generation and software fault localization are two important research objects of testing. Spectrum-based fault localization (SBFL) method is a mainstream dynamic fault localization method, which inputs test cases into the program for execution and collects statement coverage information and execution results. By analysing this information, a fault report can be generated. The performance of SBFL is affected by test case quality. Therefore, this paper proposes a test case generation method based on improved genetic algorithm to assist SBFL. This method uses a small number of initial test cases to execute the program, then uses the XGBoost model to pre-process the spectrum information and generates a list of potential fault statements to guide the iterative process of the genetic algorithm. The experimental results show that the test cases generated using the improved genetic algorithm have certain advantages over the test cases generated by general genetic algorithm and the baseline test cases.

The architectural complexity of advanced storage controllers has increased to a point at which architectural verification methods based on document inspections and reviews are no longer effective. To facilitate the architectural verification process, a high-performance simulator (TurboSim) has been developed for architectural-level verification. The TurboSim application includes a set of architectural-level models, which represent essential architectural components, and an automatic test case generator (ATG). The TurboSim ATG is used to generate realistic representations of customer direct access storage device (DASD) track data. The track data are used to drive different TurboSim simulation scenarios. After demonstrating its effectiveness as an architectural verification tool, TurboSim was enhanced to support the automatic generation of hardware test cases. These hardware test cases are used to ensure that the hardware implementation matches architectural specifications.

This paper describes a research result on automatic generation of abstract test cases from INAP protocol in formal specifications by using and applying many existing related algorithms and techniques such as the testing framework, rural Chinese postman tour and UTO (Unique Input Output) sequence concepts. We use the I/OFSM (Input/Output Finite State Machine) generated from SDL specifications and a characterizing sequence, called Unique Input/Output (UIO) sequence, is defined for the I/O FSM. The UIO sequence is combined with the concept of rural Chinese postman tour, without reliable reset capability, to obtain an optimal test sequence. To develop this method, first of all, we establish a framework leading to formal testing methodology which can be used to assess the conformance of an IUT (Implementation Under Test) to the behaviours specified in an I/OFSM which is equivalent to corresponding FDT (Formal Description Technique) specification. By this framework, we present a methodology for automatic test generation from FDT specifications. A prototype of the proposed framework has been built with special attention to real application in order to generate the executable test cases in an automatic way.

Fault in a software program can be detected by mutation testing. However, mutation testing is an expensive process in a software testing domain. In this paper, we have introduced a method based on genetic algorithm and mutation analysis for unit testing process. Software industry produces high quality software in which software testing has an important role. First, we make a program/software and intent some mutant in this program/software, find most critical path and optimise test cases using genetic algorithm for the unit testing. Initially generated test cases are refined using genetic algorithm. We use a mutant function for measuring the adequacy of the test case set. The given mutant function is used to calculate a mutant score. We have achieved 100% path coverage and boundary coverage using mutation testing. The objective is to produce a set of good test cases for killing one or more undesired mutants and produces different mutant from original software/program. Unlike simple algorithms, genetic algorithms provide suitability for reducing the data generation at a comparable cost. An optimised test case has been generated by proposed approach for cost reduction and revealing or killing undesired test cases.

Fault in a software program can be detected by mutation testing. However, mutation testing is an expensive process in a software testing domain. In this paper, we have introduced a method based on genetic algorithm and mutation analysis for unit testing process. Software industry produces high quality software in which software testing has an important role. First, we make a program/software and intent some mutant in this program/software, find most critical path and optimise test cases using genetic algorithm for the unit testing. Initially generated test cases are refined using genetic algorithm. We use a mutant function for measuring the adequacy of the test case set. The given mutant function is used to calculate a mutant score. We have achieved 100% path coverage and boundary coverage using mutation testing. The objective is to produce a set of good test cases for killing one or more undesired mutants and produces different mutant from original software/program. Unlike simple algorithms, genetic algorithms provide suitability for reducing the data generation at a comparable cost. An optimised test case has been generated by proposed approach for cost reduction and revealing or killing undesired test cases.

A new heuristic algorithm is proposed by this paper, on multi-objective optimization using metaheuristics and TSP (travelling salesman problems). Basic thinking behind this algorithm is minimizing the TSP path or tour by dividing the entire tour into blocks that are overlapped to each other and then improve each individual block separately. Although it is unproven that a good solution have small improvement chances if a node moved far way to a position compared to its original solution. By intensively searching each block, further improvement is possible in TSP path or tour that never be supported in various search methods and genetic algorithm. Proposed algorithm and computational experiment performance was tested, and these tests are carried out with support of already present instances of problem. According to the results represented by paper, the computation verifies that proposed algorithm can solve TSPs efficiently. Proposed algorithm is then used for selecting optimal test cases, thousands of those test cases which are selected after confirming that they identify bugs and they itself selected from a repository of test cases; these thousand test cases are those test cases which are selected from several thousand test cases because they detect bugs. Few test cases from repository act as milestones (nodes) and having certain weight associated with each, proposed algorithm based on TSP implemented over selected result and select the optimal result or path or solution. These selected optimal test cases or selected path are further used to perform the regression testing, by applying those test cases selected by proposed algorithm in order to remove most of the faults or bugs effectively, i.e. take less time and identify almost all the bugs with few test cases. Hence this proposed algorithm assures most effective solution for regression testing test case selection.

A primary issue in black-box testing is how to generate adequate test cases from input domain of the system under test on the basis of user's requirement specification. However, for some types of systems including embedded systems, developing test cases from output domain is more suitable than developing from input domain, especially, when the output domain is smaller. This approach ensures better reliability of the system under test. In this paper, the authors present a new approach to automate the generation of test cases from output domain of a pilot project "Temperature Monitoring and Controlling of Nuclear Reactor System" (TMCNRS) which is an embedded system developed using modified Cleanroom Software Engineering methodology. An Automated Test Case Generator (ATCG) that uses Genetic algorithms (GAs) extensively and generates test cases from output domain is proposed. The ATCG generates test cases which are useful to conduct pseudo - exhaustive testing to detect single, double and several multimode faults in the system. The generator considers most of the combinations of outputs, and finds the corresponding inputs while optimizing the number of test cases generated. In order to investigate the effectiveness of this approach, test cases were generated by ATCG and the tests were conducted on the target embedded system at a minimum cost and time. Experimental results show that this approach is very promising.