AUTOMATIC TESTING FRAMEWORK BASED ON SERENITY AND JENKINS AUTOMATED BUILD

In software testing process, test data generation represents an important step for high quality software, even for mobile devices. As proposed in previous works, a potential source for random data generation is represented by the UI layout files that are used for almost all mobile platforms (Android, iOS, Windows Phone/Mobile). This paper continues the previous work and presents a test data generation system based on Android layout files. The test data generator uses DSL files as input and generates test data that conform to several testing principles. The generated test data could be stored in XML files or any format required by the testing frameworks.

Software release is highly dependent upon the quality of the product which must be met by the product before software release. Software product quality outcome will be good if modeling of test process is good. In Software test process model large numbers of state variables exist which can affect the entire software testing process. This paper introduces various state variables for improving the software test process. By introducing as many state variables, definitely we can enhance the overall quality of testing process which ultimately will enhance the quality of software product. This paper proposes an algorithm focus upon the use of the state variables along with the control based system for the software testing process. Combine concept of controls based system and state variables provide the power of identifying the remaining number of errors at any point of time. Proposed algorithm uses the concept of the feedback system and related work to [9] [10] [12] [13] with additional state variables to model an efficient software testing process.

Software reliability testing refers to various software testing activities that are driven to achieve a quantitative reliability goal given a priori or lead to a quantitative reliability assessment for the software under test. In this paper we develop a modeling framework for the software reliability testing process, comprising a simplifying model and a generalized model. In both models the software testing action selection process and the defect removal mechanism are explicitly described. Both the discrete-time domain and the continuous-time domain are involved. The generalized model is more accurate or realistic than the simplifying model since the former avoids the assumption that defects are equally detectable and the assumption that defects are removed upon being detected. However simulation examples show that the simplifying model really captures some of essential features of the software testing process after a short initial testing stage. The modeling framework is practically realistic, mathematically rigorous, and quantitatively precise. It demonstrates that the relationship between software testing and delivered software reliability, which was poor understood, can well be formulated and quantified. Rigorous examinations show that several common assumptions adopted in software reliability modeling, including the independence assumption, the exponentiality assumption, and the NHPP assumption, are theoretically false in general. This paper sets a good starting point to further formalize and quantify the software testing process and its relation to delivered software reliability.

Automated and systematic testing of automation systems (AS) and production systems (PS) require an integrated testing tool chain for test case development, execution and reporting. In practice, the test automation tool chain cannot be fully automated because of missing links between different tools used in the test automation process. Closing these gaps typically require (high) human effort. Furthermore, domain and software testing expertise is often bundled by one (expensive) engineer who is responsible for the application domain (reflected in use cases and test cases) and software tests (software test code). This paper presents a flexible Testing Automation Framework (TAF) that enables the configuration of test processes involving different tools and various layers for test automation and enables separated roles for the application domain and software tests. We build on best-practice test automation from Software Engineering and design a test automation process for the automation systems domain. We demonstrate the feasibility with a use case, derived from production systems automation, with selected tools covering all test automation layers. First results showed the feasibility of the framework in the evaluation use case making test processes more flexible and automated. Although the successful implementation of the TAF can support the efficient configuration and execution of test processes, there is additional effort for preparing the flexible and automated tool chain.

Incomplete, ambiguous, or rapidly changing requirements can have a profound impact on the quality and cost of software development. In an effort to provide a more rigorous approach to flight-critical system development, Rockwell Collins used a formal specification modeling approach to develop the mode control logic of a Flight Guidance System (FGS) for a General Aviation class aircraft Rockwell Collins later used an early version of Test Automation Framework (TAF) approach for model-based analysis and test automation to analyze the requirement model and generate tests for a new implementation of the FGS system The TAF approach integrates various government and commercially available model development and test generation tools to support defect prevention and automated testing of systems and software. This paper describes the TAF model-based verification approach. It summarizes the new model and implementation errors that have been discovered. It briefly describes how the TAF approach can be used to locate requirement defects early in the development process, reduce manual test development effort, and reduce rework. It describes how the use of model-based development and test automation can be effectively used in the development and verification of systems that must meet the highest standards of safety, reliability, and quality.

Today, the most vital step is software testing, especially in product quality. Agile software development is exceedingly difficult and has contentious issues in literature and business. Since most Agile techniques place more emphasis on development than software testing activities, different people have varying opinions about how software testing fits into these methodologies. Agile places a high emphasis on frequent deliveries and tight client connections. However, software testing is complex because agile does not incorporate many destructive testing techniques typically required for a product of higher-quality. The study reveals industry-wide Agile testing procedures and the main problems that arise when Agile development is used. Additionally covered are the differences between manual and automated testing, best automation practices, and ways to lead different testing teams in an agile testing environment. This study covers every facet of the Agile software testing process. Agile development has shown that testing is not a distinct stage but rather a crucial component of software development along with the coding. The agile team employs an overall strategy to produce software products of the highest quality. As part of an agile team, testers collect customer samples of the desired behaviour and collaborate with the developer's team to help the coders translate these examples into executable specifications. Coding and testing are done iteratively and incrementally, developing every item until it is valuable enough to release in production. It addresses several testing modalities. The classification provided by its quadrants aids teams in determining and organizing the necessary tests. To increase quality assurance, this article will examine various and improved Agile Testing Processes and methodologies. Additionally, the topic of Agile development's software testing method is covered. Agile development processes can be improved and more advantageous by incorporating testing procedures.

Background: The development of technology, both hardware and software, is rapidly advancing due to the growth of Artificial Intelligence (AI), which simplifies many human activities, including the software testing process. This research journal discusses how software testing is conducted through a Quality Management System (QMS). Objective: The journal examines whether software testing processes are related to QMS, how significantly QMS supports software testing, and the differences between testing with QMS and without QMS. Methods: This study outlines the stages of software testing through QMS, such as Software Quality Assurance (SQA), Quality Control (QC), Test Maturity Model Integration (TMMI), and performance indicators. According to the American Society for Quality (ASQ), QMS is a formal and documented system consisting of policies, processes, and procedures designed to achieve an organization's quality policies and objectives. Results: Several experts also define QMS as a system that focuses on improving customer satisfaction, process consistency, efficiency, and the overall quality of products or services. Software, according to Roger S. Pressman, is a set of instructions (programs), data structures that allow information manipulation, and related documentation describing operations and functions. Conclusion: Overall, QMS not only improves software product quality but also strengthens the testing process, making it more optimal, efficient, and aligned with organizational goals. The integration of QMS frameworks such as ISO 9001:2015, CMMI, and TMMI provides organizations with a competitive advantage by ensuring consistent delivery of high-quality software products that meet or exceed customer expectations.

Here are obstacles in the process of making software at PT. Hartono Technology Palace. At that time, every time a test was carried out on the software being developed, the company would find different results. This causes ineffective and inefficient software development processes. Problems at PT. Hartono Istana Teknologi was found because of the less use of standard software testing methods. That is why the test results are constantly changing, and the software development process cannot be completed quickly. This problem can be solved by applying the software testing method. This research will try to solve the problem by using 2 methods, trial and error, and integration. These two methods will be compared to determine which method is more effective in the software testing process at PT. Hartono Technology Palace. Several test scenarios are also designed to be applied in this study. As a final result, the trial and error method is the most effective method to be applied at PT. Hartono Istana Teknologi. The trial and error method is considered effective because the testing process is more thorough and in accordance with the established concept. While in the integration method, the testing process only focuses on testing data without paying attention to existing functions

This paper investigates automatic testing of medium voltage drive controller using OPC server interface. OPC is a standard software interface that allows Windows program to communicate with industrial hardware devices. The control board of medium voltage drive consists of digital input signals, which are used to control the drive e.g. drive start, stop and analog signals for giving speed or torque reference to the motor. In field these signals are given through another controller e.g. PLC that serve as a master. These signals can also be generated manually using control panel. In automatic testing of the drive application software, these control signals are to be accessed by Automatic Testing Framework. Using OPC server interface in Visual Studio solution it is possible to access the PLC symbol variables and control the drive for application software functionality testing. PLC symbol variables are accessed by Automatic Testing Framework through OPC gateway server. This paper describes the hardware setup, environment used for MV drive under test and automatic testing of control software using OPC Server Interface. Test results provided here proves the advantage of using OPC interface over conventional way of testing.

Testing can be conducted with either manually or automatically. It is very difficult to perform repetitive manual tests for functional correctness, especially in large-size business applications based on client-server architecture such as trading systems in stock exchanges and banking applications, because many different test scenarios are needed to test such systems. Conducting these test scenarios manually and on a human basis within a certain time is a time consuming, erroneous and tedious process. For this reason, end to end automation (test case generation, execution and error reporting) in repetitive manual tests will increase efficiency in software testing. In this context, an approach that generates automatically test scenarios for applications based on the client-server architecture communicating with each other using Application Programming Interface (API) messages is proposed in this study.In this paper, the message-driven test case generation framework is proposed for client-server architecture systems which are communicated with each other by using API messages. Basically proposed framework automatically generates test cases, which are extracted with reverse engineering mechanism from network packets between client-server communications over networks. The studying is motivated due to the fact that there is a lack of test automation framework in a specific domain: the system based on client-server architecture which uses API messages for the communication, such as financial applications, trading systems. Our approach will ensure the reuse of test scenarios, and will ensure 100% automation in the test process from the automatic generation of the test scenarios to the execution by future work.

This work proposes a generalized excel based test automation framework for REST API testing using groovy script. Testing is an important process in software development process, which helps to find unknown errors, failure and bugs in a software application and increases quality of the software application. With the existing process of testing is manual, where the inputs are given manually and the actual outcome or output is validated for different test case scenarios. This manual testing leads to increase in time and effort of testing, as there will be different scenarios to be tested in which different test data has to be prepared to test a particular functionality. As the functionalities in an application changes or added up, over the software development phase, there is a need to test the old functionalities after and before, integrating the new or changed functionality to the existing system. This results in testing the already tested functionalities to test again manually, which increases effort and time for the testing process. The manual testing process has to be automated, so that the testing time, effort and money can be reduced. The test automation framework is implemented using a groovy compiling tool in which the functionalities such as, read the data from input test data excel file, make a REST API request, validate the data and writing the result to the excel file has been developed. Since groovy is a flavor of java, several libraries have been imported to implement the test automation framework.

Automation testing is essential to carry out functional testing quickly and precisely. Software testing is beneficial for testers doing many testing processes according to the existing scenarios. So, there is an urgency to explore a more advanced solution to replace the traditional approach. Silk Test Workbench is a tool for automation testing that can accelerate the functional testing of complex applications. For this research, we proposed automation testing on website applications with various testing scenarios that have been made. The visual test recording method is used for automation testing with Silk Test Workbench. Silk Test Workbench can speed up testing time, make a good test asset testing scenario, make it easier for users to do testing, and speed up the process to get the result in the form of a screen capture of the testing process. A comparison of testing time has been performed. The automatic testing time with the scenarios is 2 hours 46 minutes. The automatic testing time will be 2 hours 36 minutes using the all-in-one scenario. This study also used questionnaires from colleagues in the Quality Assurance field to get maximum User Acceptance Test results.

Context and background: In Production Systems Engineering (PSE), software and systems testing are success-critical along the production automation life cycle to identify defects early and efficiently. Although test automation concepts enable continuous integration and tests during engineering and maintenance, tool chains are often hardwired, less flexible, and inefficient. Thus, there is a need for more flexible tool chains to support verification and validation of control code variants. Objective: In this book chapter, we (a) describe a flexible Test Automation Framework (TAF) to enable continuous integration and tests and (b) provide an adapted maintenance process to enable efficient verification and validation of control code variants. Method: We build on best practices from Software Engineering and Software Testing to establish a flexible TAF based on Behavior-Driven Testing. We use the Abstract Syntax Tree (AST) as foundation for human-based verification and validation. We developed an initial prototype derived from industry partners and used an Industry 4.0 Testbed for evaluation. Results and conclusion: First results of the prototype implementation with selected testing tools showed the capability of the TAF concept for supporting flexible configurations of testing tool chains. The AST concept can support the human-based verification and validation of control code variants.

This paper proposes framework based on Black Box Testing methodology which is used for automatic testing of Set-top boxes (STB). Proposed framework for automatic testing can be used for functional testing, regression testing, performance testing and stress testing of STB. Complete system for automatic testing consists of web tool responsible for requirement and test management along with stand-alone application for execution of automatic test cases in form of scripts written in Python programming language. Paper gives an overview of the whole system for automatic testing along with list of framework functions and their usage for creating automatic test cases.

Software testing is a process for insuring the delivery of an error free software to customers. The success or failure of the software to achieve its desirability is depend on the software testing components. All the software industries spend a lot of their money and time for software testing process. Automatic testing is a key area where cost and time of software testing process has been considerably reduced as compared to manual testing. Path testing is a process of finding all du-path coverage for a given program. In this paper genetic algorithm is used for automatic test cases generation and optimization. The proposed method with genetic algorithm accepts a set of inputs and check of for path coverage.