Paladin

Background & Problem Statement - Software testing is a critical phase in the software development lifecycle (SDLC), ensuring that applications function correctly, meet user requirements, and maintain high- quality standards. Traditional software testing approaches, including manual testing and rule-based automation, often face challenges in scalability, efficiency, and adaptability to dynamic software environments. Traditional testing methods are overwhelmed by complex software systems which slows down defect detection and extends both testing costs and release schedules. Machine Learning (ML) has emerged as a transformative solution, introducing predictive and adaptive capabilities that optimize test case selection, automate defect detection, and enhance overall software quality assurance (QA). This study explores the integration of ML in software testing, addressing the challenges of traditional QA methodologies and demonstrating how AI-driven frameworks improve testing efficiency. Methodology - To investigate the impact of ML in software testing, this research adopts a systematic approach by analyzing ML-driven test automation techniques, including predictive testing, adaptive test execution, and automated test case generation. Research reviews how Google Microsoft Facebook IBM and Deep Code put ML-based quality assurance frameworks into operation. The study leverages supervised learning, reinforcement learning, deep learning, and NLP-based techniques to demonstrate how ML models predict software defects, dynamically adapt test cases, and optimize testing resources. The research tests how ML-based testing models operate within CI/CD pipelines to improve ongoing testing and deployment flow. Analysis & Results - The analysis of ML-driven software testing reveals that predictive analytics improves early defect detection rates. It helps developers spend 37% less time debugging their work. Adaptive testing models, including self-healing test scripts, minimize maintenance costs by 50% and enhance test reliability in agile environments. The integration of NLP-based test case generation increases test coverage. NLP technology enables automatic connection between requirements and test cases at 89% success rate. Additionally, reinforcement learning techniques improve test case selection, reducing redundant test executions by 43%. Our research shows different ML methods work well to lessen incorrect error alerts. ML integration for QA surely increasing defect prediction accuracy and optimizing test execution time. Findings & Contributions - This research contributes to the field of AI-driven software testing by providing a comprehensive framework for ML-based QA methodologies. Our study shows that machine learning helps find more software problems better adapts test cases and lowers testing expenses to solve present software development needs. The study also identifies critical challenges, including data availability, model interpretability, and computational overhead, suggesting future research directions in Explainable AI (XAI), hybrid AI-ML testing models, and AI-driven security testing. As the industry moves toward AI-first software testing, this research paves the way for fully autonomous QA frameworks, enabling intelligent, scalable, and cost- effective software validation techniques. Keywords - Machine Learning, Software Testing, Quality Assurance, Predictive Testing, Adaptive Testing, Test Automation, Defect Prediction, Self- Healing Test Scripts, AI-Driven QA, Reinforcement

Based on our experience of collaborating with industry, we observed that test case generation usually relies on test case specifications (TCSs), commonly written in natural language, specifying test cases of a System Under Test at a high level of abstraction. In practice, TCSs are commonly used by test engineers as reference documents to perform these activities: 1) Manually executing test cases in TCSs; 2) Manually coding test cases in a test scripting language for automated test case execution. In the latter case, the gap between TCSs and executable test cases has to be filled by test engineers, requiring a significant amount of coding effort and domain knowledge. Motivated by the above observations from the industry, we first propose, in this paper, a TCS language, named as Restricted Test Case Modeling (RTCM), which is based on natural language and composed of an easy-to-use template, a set of restriction rules and keywords. Second, we propose a test case generation tool (aToucan4Test), which takes TCSs in RTCM as input and generates either manual test cases or automatically executable test cases, based on various coverage criteria defined on RTCM. To assess the applicability of RTCM, we manually modeled two industrial case studies and examined 30 automatically generated TCSs. To evaluate aToucan4Test, we modeled three subsystems of a Video Conferencing System developed by Cisco Systems, Norway and automatically generated executable test cases. These test cases were successfully executed on two commercial software versions. In the paper, we also discuss our experience of applying RTCM and aToucan4Test in an industrial context and compare our approach with other model-based testing methodologies.

Software systems are not considered complete unless properly tested and verified. In existing literature, a growing interest on establishment of automated testing techniques has been observed. However, tedious manual process of test case generation largely depends upon domain knowledge and formalized representation of user requirements. The advent of semantic web engineering has led the foundation for developing ontologies as a mean to express information and knowledge semantics regarding particular domain efficiently. In software testing, ontologies can be significantly helpful to automate testing phase as they encode domain knowledge in machine interpretable format. We have proposed automatic test case generation framework that involves ontology-based requirement specification and learning based methods for conducting black box testing. Our approach integrates knowledge-based system (ontology) with learning-based testing algorithm to automate: generation of test cases, test execution and test verdict construction. Proposed framework includes, requirement ontology to formalize requirement specification, Dialogue Manager that enables selection of available test cases and Learning Based Testing to generate counter examples of test cases through system learning. The contribution of this paper is to enable 1) requirement elicitation and specification using ontologies 2) test data selection from existing ontologies and 3) automatic test case generation from existing test cases. To represent the applicability of this research, ontology for requirement elicitation and specification is developed. Framework proposed in this research paper is an effort to provide software testing tools to save time, cost and efforts during test design phase.

Although Android apps are becoming more and more popular, apps tend to contain defects which can ultimately manifest as failures (or crashes) to end-users. In order to improve software quality, people have proposed different automated tools for testing Android applications. The automatic generation of test cases is an important means to improve the efficiency of software testing, but many tools related to test case generation do not seem to be fully adapted to the problem of Android test case generation. Therefore, this article aims to design and evaluate the automatic generation method of Android application test cases based on the multi-dimensional coverage. We conducted test experiments on 48 self-developed Android apps through the Gradle plugin, and the final average target coverage reached about 32%. Considering that the current component modules are not yet complete, we believe that tools will reach even higher coverage in the future.

Business Process Model and Notation (BPMN) is a popular and widespread modelling language used to describe business processes. These BPMN business process models can serve as a foundation for functional software testing. Functional software testing is an important part of software development, which ensures that software works as expected and that it includes all the desired functionality, as defined in the process models. This position paper presents an approach and considers two different methods on how to automatically create functional test cases from BPMN business process models. The generated test cases shall be understandable for all stakeholders and abstracting from the technical implementation. To achieve this general understandability of the test cases, Gherkin is used as a test case definition language. The two proposed methods will be developed and evaluated in future work. This planned evaluation includes comparing the automatically created test cases with manually created ones.

In software development, testing often takes more than half the total development time (Pan 1999). Test case design and execution of test procedures consume most of the testing time. Thus, automatically generating test cases and automatically detecting errors in test procedures prior to execution is highly advantageous. This thesis proposes a new approach to further automate test case design and the test procedure development process. Several open-source products exist to automate test case design, but they have limitations including test cases that do not trace back to models; test cases that are not reusable for libraries; and limiting test cases to generation on their own test environment. This limits their support for the important, new avionics standard, DO-178C (RTCA 2012). The first contribution of the thesis is a technique for test code generation that, compared to existing products, is faster, provides improved traceability to models, and supports reusable test procedures that can be generated on any testing environment. To address the current limitations, the new approach utilizes the Simulink Design Verifier and an open-source constraint solver to generate test cases. The technique allows each test case to be traced back to an expression and to the original model. Detecting errors in manually written test procedures before testing starts is also critical to efficient verification. It can save hours or even days if errors are detected in the early test procedure design stage. However, analysis done here of a set of open source code analysis tools shows that they cannot detect type and attribute errors effectively. The second contribution of the thesis is to develop a static code analyzer for Python code that detects bugs that could cause automated test procedures to crash. The analyzer converts a Python code to an abstract syntax tree and detects all type and attribute errors by

On the order of test goals in specification-based testing

Testing plays a vital role in the evolution and establishment of any quality product as well as any quality system. Testing is essential to prove the correctness (valid output when input is valid, and proper handling techniques when input is invalid) of the system and it is crucial to prove the compatibility of the system with the operating environment. In component-based software systems, various components interact with each other to access as well as provide required functionalities. In such complex systems testing is one of the most important activities. Since component-based software engineering relies on the concept of “use of pre-built and pre-tested components”, our focus here, is on functional testing rather than structural testing. Functional testing emphasizes the behavioural attributes of the components when they interact with each other. Components interact through operands and parameters. In this paper we suggest functional testing strategy and test case generation technique for component-based software. When two components are integrated then they generate some specific effect. This strategy is named Integration-effect graph. This strategy is a black-box technique as it covers the input and output domains only. We have used the graph theory notations to show the integration and interaction among the components.

Test cases are essential parts to perform software testing as they play very significant role defining the test steps to work. Test cases can be generated either manually or automatically. Creating test cases manually has several disadvantages, such as higher probability in producing inaccurate results. Furthermore, generating test cases manually is ineffective since manual methods will take longer and require high human efforts. Therefore, generating test cases automatically is more preferable to limit the previous problems. Automated testing can also provide more accurate results than manual testing because it is less prone to human errors encouraging software developers to generate test cases automatically. Test cases can be generated from various sources, including UML diagrams. In this paper, we propose a method to generate test cases automatically by utilising depth-first search algorithms. The UML diagram used in this study is an activity diagram. This paper also provides a method to parse activity diagram that is created using the visual paradigm application that has been converted into an XML format. Our method is able to parse and convert the XML format into graph or tree form. Using our proposed method, a test case search will be carried out using the depth-first search on the tree form to generate test cases. Lastly, we also evaluate our method by comparing the generated test cases with cyclomatic complexity, the method will be performed manually and automatically to validate the results of many scenarios. Our result showed that our method could generate the same scenario (100% coverage) as the manual validation method.

Automated testing techniques for software design specification include model-based approaches that generate test cases from a coding or state machine model of the interface, capture-replay methods that require the user to specify and demonstrate each test case individually, and modelling-language approaches that provide templates for abstract test cases. In this paper elucidates a method for automatic software analysis testing for IoT service. Nowadays real-time IoT software is used to satisfy the needs and functions of a user and it is assisted in this by the hardware. Software testing is generally used to detect bugs and errors and eliminate them to produce a high quality, reliable software. The automated testing depends on the kind of approach taken. This testing may be poorly implemented which results in a low quality software or it may be effectively implemented which results in a high quality software. Automated IoT software testing can decrease time and cost. When these techniques are identified, the cost and time can be reduced. The automated testing is based on three steps such as analyzing the test case, test execution and log analyzer. These steps provide user friendly software with high standard. The automated process provides test reusability and repeatability and saves the test coverage cost and time. In this paper compare the automated software testing for IoT transactions and system peripherals such as sensor, DMA controller, internal counters, physical layer, and virtual layer using IoT specification based software testing.

Software testing is the key to ensuring software quality. Test case generation and software fault localization are two important research objects of testing. Spectrum-based fault localization (SBFL) method is a mainstream dynamic fault localization method, which inputs test cases into the program for execution and collects statement coverage information and execution results. By analysing this information, a fault report can be generated. The performance of SBFL is affected by test case quality. Therefore, this paper proposes a test case generation method based on improved genetic algorithm to assist SBFL. This method uses a small number of initial test cases to execute the program, then uses the XGBoost model to pre-process the spectrum information and generates a list of potential fault statements to guide the iterative process of the genetic algorithm. The experimental results show that the test cases generated using the improved genetic algorithm have certain advantages over the test cases generated by general genetic algorithm and the baseline test cases.

Services are highly reusable, flexible and loosely coupled, which makes the evolution and the maintenance of composite services more complex. Evolution of BPEL composite service covers changes of processes, bindings and interfaces. In this paper, an approach is proposed to select and generate test cases during the evolution of BPEL composite service. The approach identifies the changes by using control-flow analysis technique and comparing the paths in new composite service version and the old one using extensible BPEL flow graph (or XBFG). Message flow is appended to the control flow so that XBFG can describe the behavior of composite service integrally. The binding and predicate constraint information added in XBFG elements can be used in path selection and test case generation. Theory analysis and case study both show that the approach is effective, and test cases coverage rate is high for the changes of processes, bindings and interfaces.

As we know, software testing is an important part of software development lifecycle. More than 50% of the whole system development work and total cost were spend on the software testing. And it’s estimated that almost 60% of the total test time and cost were spent on the design of test cases. In recent years, the automated testing gradually replaces the traditional manual testing to become an important branch of software testing. In automated testing, the most important thing is to design and generate valid test case automatically. In this paper, In order to reduce the workload of testers on the test case generation, improve the test efficiency and makes test experiences be passed on, we design a common description method for the test case based on ontology; propose a semantic similarity measure method to retrieve the usable test cases from test case library based on WordNet; establish the relationship between the test case and test requirement though rules to modify the retrieved test case; generate the final test case sequence. At last, the practical applicability of the approach is evaluated through an experiment. Keywords-software testing; ontology; test case generation; semantic similarity measure

While automated testing of mobile applications is very useful for checking run-time behaviours and specifications, its capability in discovering issues in apps is often limited in practice due to long testing time. A common practice is to randomly and exhaustively explore the whole app test space, which takes a lot of time and resource to achieve good coverage and reach targeted parts of the apps.In this paper, we present MAMBA1, a directed testing system for checking privacy in Android apps. MAMBA performs path searches of user events in control-flow graphs of callbacks generated from static analysis of app bytecode. Based on the paths found, it builds test cases comprised of user events that can trigger the executions of the apps and quickly direct the apps’ activity transitions from the starting activity towards target activities of interest, revealing potential accesses to privacy-sensitive data in the apps. MAMBA’s backend testing engine then simulates the executions of the apps following the generated test cases to check actual runtime behavior of the apps that may leak users’ private data. We evaluated MAMBA against another automated testing approach that exhaustively searches for target activities in 24 apps, and found that our graph-aided directed testing achieves the same coverage of target activities 6.1 times faster on average, including the time required for bytecode analysis and test case generation. By instrumenting privacy access/leak detectors during testing, we were able to verify from test logs that almost half of target activities accessed user privacy data, and 26.7% of target activities leaked privacy data to the network.

This work aims at defining a procedure and a set of mechanisms able to improve the quality of the code coverage in automated software reverse engineering processes, and specifically in automated GUI-driven testing of Android apps. Existing automated model-based testing techniques, based on reverse engineering, generate test cases which can be executed directly on the software's GUI. We propose to augment the code coverage of these techniques, by exploiting information from patterns, defined at different levels (application design, state-based model, interaction with Android services), and generating additional test cases that may increase the coverage capability of GUI-Ripping based testing technique. The generation of the additional test cases is accomplished by defining an automatable procedure which exploits an existing GUI testing approach and a pattern based approach used in a different context.