Automated generation of testcase datasets

Testing mobile application refers to the process of ensuring the quality and functionality of the software applications implemented for mobile phones. It is a fundamental step to test mobile applications specially before being released to the market. Load testing is a kind of software testing which is carried out to test the performance of mobile applications under different workloads. However, most of mobile apps are implemented within a bounded resource, budget, and time due to the market pressure. This extremely influences the quality of the mobile applications development industry. The automatic test case generation is the key factor for efficient and cost-effective software verification process. Model-based load testing represents both performance aspects and functional aspects of mobile applications by using modeling languages like unified modeling language (UML). Then, adopting the resulting models to automatically design test cases that will be used for load testing. This paper proposes an automatic model-based test case generation approach for mobile applications load testing. The proposed approach lessens testing time and satisfies fast time-to market constraint of mobile apps. Additionally, the proposed approach validates the mobile applications functionalities as well as the performance requirements.

This work aims to generate White-box Test Cases from a mainframe NATURAL code fragment, using the Control Flow Graph Technique. Basically, it enables a code fragment analysis, generating its control flow represented by a Graph perspective. As a consequence, it provides the automatic generation of White-box Test Cases. Furthermore, this work contributes to attain a lower degree of difficult in the execution of White-box Tests within a Mainframe environment. Also, it brings a significant contribution related to the execution time, inherent to the software testing. At the end, it is added a testing expertise to NATURAL development teams. Usually there is not enough available time to test all possible paths in a algorithm, even in a simple application. Based on it, this work provides automatic generation of test cases for empowerment of team decision, regarding which test cases are more relevant to be performed.

At present, the design of the test process during the manufacturing process of the aircraft is mainly performed by the process personnel to consult the relevant data of the tested object to obtain the test information, and manually write the test process file. Therefore, the writing efficiency is low and the writing quality relies heavily on the experience of craftsmen, and the standardization and standardization of process files cannot be effectively guaranteed. On the other hand, the test program is closely coupled with specific test equipment. The upgrade and update of test resources cause the test program to change, which increases the use and maintenance costs.This paper proposes to implement standardized modeling of test knowledge based on the Automatic Test Markup Language (IEEE 1671 ATML), to support the sharing, transplantation and reuse of the test information, and to standardize and digitize test process files. At the same time, the automatic generation of signal-oriented test cases is achieved, the interchangeability of test instruments and the portability of test programs are solved, and the universal execution of various test resources is achieved.

This paper describes a research result on automatic generation of abstract test cases from INAP protocol in formal specifications by using and applying many existing related algorithms and techniques such as the testing framework, rural Chinese postman tour and UTO (Unique Input Output) sequence concepts. We use the I/OFSM (Input/Output Finite State Machine) generated from SDL specifications and a characterizing sequence, called Unique Input/Output (UIO) sequence, is defined for the I/O FSM. The UIO sequence is combined with the concept of rural Chinese postman tour, without reliable reset capability, to obtain an optimal test sequence. To develop this method, first of all, we establish a framework leading to formal testing methodology which can be used to assess the conformance of an IUT (Implementation Under Test) to the behaviours specified in an I/OFSM which is equivalent to corresponding FDT (Formal Description Technique) specification. By this framework, we present a methodology for automatic test generation from FDT specifications. A prototype of the proposed framework has been built with special attention to real application in order to generate the executable test cases in an automatic way.

The architectural complexity of advanced storage controllers has increased to a point at which architectural verification methods based on document inspections and reviews are no longer effective. To facilitate the architectural verification process, a high-performance simulator (TurboSim) has been developed for architectural-level verification. The TurboSim application includes a set of architectural-level models, which represent essential architectural components, and an automatic test case generator (ATG). The TurboSim ATG is used to generate realistic representations of customer direct access storage device (DASD) track data. The track data are used to drive different TurboSim simulation scenarios. After demonstrating its effectiveness as an architectural verification tool, TurboSim was enhanced to support the automatic generation of hardware test cases. These hardware test cases are used to ensure that the hardware implementation matches architectural specifications.

An alternative scheme to formal software specification is explicitly proposed. In our approach, a formal software specification is formally defined as a set of decision rules performed by a software system. We propose a set of graphical notations called Requirements Particle Network (RPN), to describe the essential preconditions and operations needed by a software system according to the decision rules. A RPN consists of a set of particles and edges to construct a visual model of a decision rule during the software analysis phase. In addition, a number of transformation rules are proposed to perform the formal specification synthesis. A set of predefined formal requirements particle definitions is written in prior and reused during the transformation steps. A developer is provided a practical mean to write a formal specification with a brief experience in mathematical background. In this research, we demonstrate the Z formal specification synthesis using RPN. The usability of the RPN approach is investigated by conducting a workshop. The result indicates that a developer with experience in writing data flow diagram is capable to produce a complete and consistent RPN. Moreover, we show a case study of applying RPN to construct a composite operation to be used in database applications

This report gives an overview of the current status and plans to construct a prototype environment of advanced tools for software and hardware development based on the use of wide-spectrum languages. The wide-spectrum languages include Anna (ANNotated Ada), and TSL (Task Sequencing Language). The tools described here provide interactive aid at all stages in the system development process. Special emphasis is placed on distributed computing, both in providing tools for handling parallelism in the subject system, and in designing tools that utilize parallelism in the programming environment. Applications of these tools include requirements analysis, formal specification, rapid prototyping, testing, formal verfication and construction of self-testing Ada software for multi-processor systems.The report describes an existing environment of prototype tools supporting applications of Anna and TSL to formal specification and testing of Ada software. The new environment tools will be based on component tools already developed at Stanford and proven to be portable to various Ada environments. All tools are implemented in Ada and are intended to interface with standard components of Ada programming environments.

Although Android apps are becoming more and more popular, apps tend to contain defects which can ultimately manifest as failures (or crashes) to end-users. In order to improve software quality, people have proposed different automated tools for testing Android applications. The automatic generation of test cases is an important means to improve the efficiency of software testing, but many tools related to test case generation do not seem to be fully adapted to the problem of Android test case generation. Therefore, this article aims to design and evaluate the automatic generation method of Android application test cases based on the multi-dimensional coverage. We conducted test experiments on 48 self-developed Android apps through the Gradle plugin, and the final average target coverage reached about 32%. Considering that the current component modules are not yet complete, we believe that tools will reach even higher coverage in the future.

We propose a complete methodology for the automatic generation of test cases in the context of digital circuit validation. Our approach is based on a software model of the system to verify in which some modules are written in the Esterel language. An initial test suite is simulated and the state coverage is computed. New test sequences are automatically generated to reach the missing states. We then convert those sequences into system-level test cases (i.e. instruction sequences) by a technique called pipeline inversion. The method has been applied for the functional validation of an industrial DSP system giving promising results.

Describes research and development into techniques to support the automatic generation of test cases for event-oriented, real-time embedded systems. A consistent suite of test scenarios can assure consistency at all levels of design activities. Although we have developed algorithms that are designed to generate test scenarios from a state-based functional requirements model, their applicability is severely limited without a means to automatically translate the model functions into a form that can be readily integrated with the algorithms. A method and tool are presented that extract the requirements model and synthesize an equivalent high-level functional representation. The tool, called the Requirements Model Code Synthesizer, has been applied to a number of design cases, one of which is described in this paper.

Several techniques and tools have been proposed for the automatic generation of test cases. Usually, these tools are evaluated in terms of fault-revealing or coverage capability, but their impact on the manual debugging activity is not considered. The question is whether automatically generated test cases are equally effective in supporting debugging as manually written tests. We conducted a family of three experiments (five replications) with humans (in total, 55 subjects) to assess whether the features of automatically generated test cases, which make them less readable and understandable (e.g., unclear test scenarios, meaningless identifiers), have an impact on the effectiveness and efficiency of debugging. The first two experiments compare different test case generation tools (Randoop vs. EvoSuite). The third experiment investigates the role of code identifiers in test cases (obfuscated vs. original identifiers), since a major difference between manual and automatically generated test cases is that the latter contain meaningless (obfuscated) identifiers. We show that automatically generated test cases are as useful for debugging as manual test cases. Furthermore, we find that, for less experienced developers, automatic tests are more useful on average due to their lower static and dynamic complexity.

Any Instrumentation and Control (I&C) system has to be properly tested to minimize the risk of faults in the system. This is even more crucial if the I&C system implements a safety function. The function of the I&C is often designed in a form of Function Block Diagram (FBD). The presented approach describes a new analytic method for automatic generation of test cases for FBD models. The test cases are generated in order to verify that the implementation behaves as intended. The test cases are designed to be used for gray-box testing of the implemented model. The automatic generation of test cases is generally faster and less prone to miss out an unsafe scenario than manual design.

Automatic generation of test cases from Boolean specifications using the MUMCUT strategy

Software systems are not considered complete unless properly tested and verified. In existing literature, a growing interest on establishment of automated testing techniques has been observed. However, tedious manual process of test case generation largely depends upon domain knowledge and formalized representation of user requirements. The advent of semantic web engineering has led the foundation for developing ontologies as a mean to express information and knowledge semantics regarding particular domain efficiently. In software testing, ontologies can be significantly helpful to automate testing phase as they encode domain knowledge in machine interpretable format. We have proposed automatic test case generation framework that involves ontology-based requirement specification and learning based methods for conducting black box testing. Our approach integrates knowledge-based system (ontology) with learning-based testing algorithm to automate: generation of test cases, test execution and test verdict construction. Proposed framework includes, requirement ontology to formalize requirement specification, Dialogue Manager that enables selection of available test cases and Learning Based Testing to generate counter examples of test cases through system learning. The contribution of this paper is to enable 1) requirement elicitation and specification using ontologies 2) test data selection from existing ontologies and 3) automatic test case generation from existing test cases. To represent the applicability of this research, ontology for requirement elicitation and specification is developed. Framework proposed in this research paper is an effort to provide software testing tools to save time, cost and efforts during test design phase.

In the past decades, automated test case generation for detecting software vulnerabilities using the search-based algorithm has been a crucial research area for security researchers. Several proposed techniques partially achieve the automatic test case generation of detecting security issues that target specific programming languages or API types. This review paper performs a comparative analysis of research papers published during 2010–21 specific to the search-based techniques used for automatic test case generation. We have chosen five criteria for evaluating the proposed techniques. These are target vulnerability, search-based algorithms used, test case generation source, test case generation method, and target language. We focused primarily on the techniques detecting the four most dangerous vulnerabilities found in the modern distributed system, namely Denial of Service (e.g., program crash), SQL injection, Cross-Site Scripting, and XML injection. We also incorporated the fitness function comparison table for each method, including the merits and limitations. This work will assist security researchers in finding new directions in the field of search-based software security testing.KeywordsSecurity testingSearch-based software security testing (SBSST)SBSEProgram crashDoSXMLISQLIXSS