Evolutionary Based Automated Coverage Analysis for GUI Testing

Software testing in general and graphical user interface (GUI) testing in particular is one of the major challenges in the lifecycle of any software system. GUI testing is inherently more difficult than the traditional and command-line interface testing. Some of the factors that make GUI testing different from the traditional software testing and significantly more difficult are: a large number of objects, different look and feel of objects, many parameters associated with each object, progressive disclosure, complex inputs from multiple sources, and graphical outputs. The existing testing techniques for the creation and management of test suites need to be adapted/enhanced for GUIs, and new testing techniques are desired to make the creation and management of test suites more efficient and effective. In this article, a methodology is proposed to create test suites for a GUI. The proposed methodology organizes the testing activity into various levels. The tests created at a particular level can be reused at higher levels. This methodology extends the notion of modularity and reusability to the testing phase. The organization and management of the created test suites resembles closely to the structure of the GUI under test.

This position paper presents a synopsis of the imperative function artificial intelligence (AI) has partaken in software engineering (SE) as well as in software testing. In addition, the paper discusses how graphical user interface (GUI), and event driven software testing can derive benefits from the use of AI techniques. Artificial intelligence has significantly aided the process of the automation of different software process. The employment of AI in software testing is not novel, having played a crucial role in the automation of software testing since its innovation. The usage of AI techniques not only reduces the cost but it also guarantees better quality as well as thorough testing. GUI Testing can be considered as the most challenging area of software testing. Although the results are quite preliminary, but the application of different AI techniques for GUI testing has proven to produce ideal results. Nevertheless, the application of AI techniques in GUI testing, in comparison with software testing, which has procured much assistance by venturing with AI, leaves much to be desired.

Software testing has become an important phase in software applications’ lifecycle. Graphical User Interface (GUI) components can be found in a large number of desktops and web applications and also in a wide variety of mobile devices. In the last years GUIs have become more and more complex and interactive, their testing process requiring an interaction with the GUI components, mainly by generating mouse, keyboard and touch events. Given their increased importance, GUIs verification for correctness contributes to the establishment of the correct functionality of the corresponding software application. The current research on GUI testing methodologies primarily focuses on automated testing. This paper presents EvoGUITest, a novel automated GUI testing framework based on evolutionary algorithms which tests the GUI independently from the application code itself. The framework is designed for testing GUIs of web applications. Results have been compared, based on specific metrics, with others existing frameworks.

The practitioner interested in reducing software verification effort may found herself lost in the many alternative definitions of Graphical User Interface (GUI) testing that exist and their relation to the notion of system testing. One result of these many definitions is that one may end up testing the same parts of the Software Under Test (SUT), specifically the application logic, twice. To clarify two important testing activities and avoid duplicate testing effort, this paper empirically evaluates to what extent GUI tests exercise the application logic of the software under test (and not only the GUI code). Experimental results show that GUI tests do not necessarily entirely exercise application logic functionality, at least not as much as system tests directly interacting with application logic code.

Graphical User Interface (GUI) has become one of the most significant parts of mobile applications (apps). It is a direct bridge between mobile apps and end users, which directly affects the end user’s experience. Neglecting GUI quality can undermine the value and effectiveness of the entire mobile app solution. Significant research efforts have been devoted to GUI testing, one effective method to ensure mobile app quality. By conducting rigorous GUI testing, developers can ensure that the visual and interactive elements of the mobile apps not only meet functional requirements but also provide a seamless and user-friendly experience. However, traditional solutions, relying on the source code or layout files, have met challenges in both effectiveness and efficiency due to the gap between what is obtained and what app GUI actually presents. Vision-based mobile app GUI testing approaches emerged with the development of computer vision technologies and have achieved promising progress. In this survey article, we provide a comprehensive investigation of the state-of-the-art techniques on 271 articles, among which 92 are vision-based studies. This survey covers different topics of GUI testing, like GUI test generation, GUI test record & replay, GUI testing framework, and so on. In particular, we highlight the emerging role of vision-based techniques and analyze how they reshape traditional approaches to mobile app GUI testing. Based on the investigation of existing studies, we outline the challenges and opportunities of (vision-based) mobile app GUI testing and propose promising research directions with the combination of emerging techniques.

Graphical User Interfaces (GUIs) are amongst the most common user interfaces, enabling interactions with applications through mouse movements and key presses. Tools for automated testing of programs through their GUI exist, however they usually rely on operating system or framework specific knowledge to interact with an application. Due to frequent operating system updates, which can remove required information, and a large variety of different GUI frameworks using unique underlying data structures, such tools rapidly become obsolete, Consequently, for an automated GUI test generation tool, supporting many frameworks and operating systems is impractical. We propose a technique for improving GUI testing by automatically identifying GUI widgets in screen shots using machine learning techniques. As training data, we generate randomized GUIs to automatically extract widget information. The resulting model provides guidance to GUI testing tools in environments not currently supported by deriving GUI widget information from screen shots only. In our experiments, we found that identifying GUI widgets in screen shots and using this information to guide random testing achieved a significantly higher branch coverage in 18 of 20 applications, with an average increase of 42.5% when compared to conventional random testing.

Graphical user interface (GUI) is an important feature of current software, and testing for GUI is considered as the most efficient method to guarantee the quality of GUI. It is consensus that in industry and academia automated GUI testing approach and tool have been a great help in reducing time and cost in testing, especially regression testing. However, almost all existing automated GUI testing approaches are code intrusive, and strongly dependent on platform or system environment, which leads to the lack of general applicability of GUI testing tools. And most of these tools can only be used in specific software or software platforms. To deal with the challenge, this paper proposes a novel automated GUI testing based on image recognition, which is a non-intrusive and general GUI testing approach. A platform called Auto GUI Testing Platform (AGTP) based on our approach is developed. And a case study is proposed to demonstrate the validity and practicability of AGTP.

The paper provides the design of a distributed test system for testing distributed GUI (Graphical User Interface) applications. The distributed GUI application that is considered allows multiple operators or users to interact with a distributed system in parallel. The distributed GUI application allows the use of existing GUI test frameworks for local test execution and offers additional services of coordination and synchronization among the deployed GUI test frameworks, test verdict arbitration, and debugging support, all of which are needed to deal with the concurrent nature of the system under test. In addition, the paper offers a test specification method for test cases that is based on the semantic model of a concurrent event flow graph. This method allows the automatic test case execution on top of the suggested distributed test system. The overall approach allows the extension of testing of single GUI application systems towards testing of distributed GUI application systems by preserving investments in test automation for single GUI testing.

In this paper we investigate the effect of test case length on the performance of test suites for automated graphical user interface (GUI) testing. Research has been done on the effect of test length for traditional white-box testing, however, these findings do not necessarily extend to GUI testing. We run experiments on five subject applications using a custom-built dynamic test generator. We measure the effect that test length has on statement, branch, and event-pair coverage, as well as the fault-finding abilities for each test suite. Our findings indicate that longer test suites perform better than shorter test suites in terms of coverage, fault-finding, and cost. However, the effect eventually diminishes with greater increase in length. As well, the results vary depending on the application under test.

Graphical user interfaces (GUIs) for today's applications are extremely large. Moreover, they provide many degrees of freedom to the end-user, thus allowing the user to perform a very large number of event sequences on the GUI. The large sizes and degrees of freedom create severe problems for GUI quality assurance, including GUI testing. In this paper, we leverage methods and measures from network analysis to analyze and study GUIs, with the goal of aiding GUI testing activities. We apply these methods and measures on the event-flow graph model of GUIs. Results of a case study show that "network centrality measures" are able to identify the most important events in the GUI as well as the most important sequences of events. These events and sequences are good candidates for test prioritization. In addition, the "betweenness clustering" method is able to partition the GUI into regions that can be tested separately.

This paper introduces an automated grader for Java programs called GUI_Grader that allows students a degree of flexibility in graphical user interface (GUI) design. GUI_Grader allows students to build multi-window Java applications, choose among alternative GUI components, and decide how to order, position and label components. This enables students to practice some aspects of designing their own GUI applications while still providing automated grading `based on a single test plan. The data-driven approach helps to maintain consistency between test plans and program specifications. Testing GUI_Grader on Java assignments from both a first-year and an upper-year course confirms the feasibility of the approach.

Short Paper; Topics: Verification/Validation, Planning Graphical User Interfaces (GUIs) are very popular and common in computer-based systems. Testing GUIs is, on the other hand, a difficult and challenging task for many reasons: First, the input space possesses a great, potentially indefinite number of combinations of inputs and events as system outputs and external events may interact with these inputs. Second, even simple GUIs possess an enormous number of states which are also due to interact with the inputs. Last but not least, many complex dependencies may hold between the states of the GUI system, and/or between its states and -inputs.Nevertheless, most Human-Computer-Interfaces (HCI) will be designed and implemented via GUI nowadays. There exist a vast amount of research work for specification of HCI, there has been, however, little well-known systematic study in this field, resulting an effective testing strategy which is not only easy to apply, but also scalable in sense of stepwise increasing the number of test cases, thus also increasing the test costs in accordance with the test budget of the project.

Graphical User Interfaces (GUIs) are substantial parts of today's applications, no matter whether these run on tablets, smartphones or desktop platforms. Since the GUI is often the only component that humans interact with, it demands for thorough testing to ensure an efficient and satisfactory user experience. Being the glue between almost all of an application's components, GUIs also lend themselves for system level testing. However, GUI testing is inherently difficult and often involves great manual labor, even with modern tools which promise automation. This paper introduces a Java library called GUITest, which allows to generate fully automated GUI robustness tests for complex applications, without the need to manually generate models or input sequences. We will explain how it operates and present first results on its applicability and effectivity during a test involving Microsoft Word.

Background & Problem Statement - Software testing is a critical phase in the software development lifecycle (SDLC), ensuring that applications function correctly, meet user requirements, and maintain high- quality standards. Traditional software testing approaches, including manual testing and rule-based automation, often face challenges in scalability, efficiency, and adaptability to dynamic software environments. Traditional testing methods are overwhelmed by complex software systems which slows down defect detection and extends both testing costs and release schedules. Machine Learning (ML) has emerged as a transformative solution, introducing predictive and adaptive capabilities that optimize test case selection, automate defect detection, and enhance overall software quality assurance (QA). This study explores the integration of ML in software testing, addressing the challenges of traditional QA methodologies and demonstrating how AI-driven frameworks improve testing efficiency. Methodology - To investigate the impact of ML in software testing, this research adopts a systematic approach by analyzing ML-driven test automation techniques, including predictive testing, adaptive test execution, and automated test case generation. Research reviews how Google Microsoft Facebook IBM and Deep Code put ML-based quality assurance frameworks into operation. The study leverages supervised learning, reinforcement learning, deep learning, and NLP-based techniques to demonstrate how ML models predict software defects, dynamically adapt test cases, and optimize testing resources. The research tests how ML-based testing models operate within CI/CD pipelines to improve ongoing testing and deployment flow. Analysis & Results - The analysis of ML-driven software testing reveals that predictive analytics improves early defect detection rates. It helps developers spend 37% less time debugging their work. Adaptive testing models, including self-healing test scripts, minimize maintenance costs by 50% and enhance test reliability in agile environments. The integration of NLP-based test case generation increases test coverage. NLP technology enables automatic connection between requirements and test cases at 89% success rate. Additionally, reinforcement learning techniques improve test case selection, reducing redundant test executions by 43%. Our research shows different ML methods work well to lessen incorrect error alerts. ML integration for QA surely increasing defect prediction accuracy and optimizing test execution time. Findings & Contributions - This research contributes to the field of AI-driven software testing by providing a comprehensive framework for ML-based QA methodologies. Our study shows that machine learning helps find more software problems better adapts test cases and lowers testing expenses to solve present software development needs. The study also identifies critical challenges, including data availability, model interpretability, and computational overhead, suggesting future research directions in Explainable AI (XAI), hybrid AI-ML testing models, and AI-driven security testing. As the industry moves toward AI-first software testing, this research paves the way for fully autonomous QA frameworks, enabling intelligent, scalable, and cost- effective software validation techniques. Keywords - Machine Learning, Software Testing, Quality Assurance, Predictive Testing, Adaptive Testing, Test Automation, Defect Prediction, Self- Healing Test Scripts, AI-Driven QA, Reinforcement

Automated testing in web applications is becoming increasingly popular due to its cost-effectiveness, efficiency, and repeatability. Ensuring the quality of such software is vital, especially in scenarios where business requirements are uncertain and volatile. In this paper, we propose Cytestion, a tool that automatically explores a web application’s Graphical User Interface (GUI) to detect faults that cause visible failures, such as crashes, errors, and unexpected behaviors. For that, it applies a scriptless and progressive approach. To work as an oracle, Cytestion evaluates the output of each test case execution by considering the HTTP status of server requests, default and customized failure GUI messages, and console failure messages. We evaluated the effectiveness and cost of our approach in two empirical studies. The first study works with injected faults in open-source projects, while the second is a case study on eight industrial projects. Cytestion detected several faults in both studies and outperformed a state-of-the-art GUI testing tool.