A Graphical User Interface (GUI) Testing Methodology

In this paper we investigate the effect of test case length on the performance of test suites for automated graphical user interface (GUI) testing. Research has been done on the effect of test length for traditional white-box testing, however, these findings do not necessarily extend to GUI testing. We run experiments on five subject applications using a custom-built dynamic test generator. We measure the effect that test length has on statement, branch, and event-pair coverage, as well as the fault-finding abilities for each test suite. Our findings indicate that longer test suites perform better than shorter test suites in terms of coverage, fault-finding, and cost. However, the effect eventually diminishes with greater increase in length. As well, the results vary depending on the application under test.

The practitioner interested in reducing software verification effort may found herself lost in the many alternative definitions of Graphical User Interface (GUI) testing that exist and their relation to the notion of system testing. One result of these many definitions is that one may end up testing the same parts of the Software Under Test (SUT), specifically the application logic, twice. To clarify two important testing activities and avoid duplicate testing effort, this paper empirically evaluates to what extent GUI tests exercise the application logic of the software under test (and not only the GUI code). Experimental results show that GUI tests do not necessarily entirely exercise application logic functionality, at least not as much as system tests directly interacting with application logic code.

This position paper presents a synopsis of the imperative function artificial intelligence (AI) has partaken in software engineering (SE) as well as in software testing. In addition, the paper discusses how graphical user interface (GUI), and event driven software testing can derive benefits from the use of AI techniques. Artificial intelligence has significantly aided the process of the automation of different software process. The employment of AI in software testing is not novel, having played a crucial role in the automation of software testing since its innovation. The usage of AI techniques not only reduces the cost but it also guarantees better quality as well as thorough testing. GUI Testing can be considered as the most challenging area of software testing. Although the results are quite preliminary, but the application of different AI techniques for GUI testing has proven to produce ideal results. Nevertheless, the application of AI techniques in GUI testing, in comparison with software testing, which has procured much assistance by venturing with AI, leaves much to be desired.

Graphical User Interface (GUI) has become one of the most significant parts of mobile applications (apps). It is a direct bridge between mobile apps and end users, which directly affects the end user’s experience. Neglecting GUI quality can undermine the value and effectiveness of the entire mobile app solution. Significant research efforts have been devoted to GUI testing, one effective method to ensure mobile app quality. By conducting rigorous GUI testing, developers can ensure that the visual and interactive elements of the mobile apps not only meet functional requirements but also provide a seamless and user-friendly experience. However, traditional solutions, relying on the source code or layout files, have met challenges in both effectiveness and efficiency due to the gap between what is obtained and what app GUI actually presents. Vision-based mobile app GUI testing approaches emerged with the development of computer vision technologies and have achieved promising progress. In this survey article, we provide a comprehensive investigation of the state-of-the-art techniques on 271 articles, among which 92 are vision-based studies. This survey covers different topics of GUI testing, like GUI test generation, GUI test record & replay, GUI testing framework, and so on. In particular, we highlight the emerging role of vision-based techniques and analyze how they reshape traditional approaches to mobile app GUI testing. Based on the investigation of existing studies, we outline the challenges and opportunities of (vision-based) mobile app GUI testing and propose promising research directions with the combination of emerging techniques.

Graphical User Interfaces (GUIs) are amongst the most common user interfaces, enabling interactions with applications through mouse movements and key presses. Tools for automated testing of programs through their GUI exist, however they usually rely on operating system or framework specific knowledge to interact with an application. Due to frequent operating system updates, which can remove required information, and a large variety of different GUI frameworks using unique underlying data structures, such tools rapidly become obsolete, Consequently, for an automated GUI test generation tool, supporting many frameworks and operating systems is impractical. We propose a technique for improving GUI testing by automatically identifying GUI widgets in screen shots using machine learning techniques. As training data, we generate randomized GUIs to automatically extract widget information. The resulting model provides guidance to GUI testing tools in environments not currently supported by deriving GUI widget information from screen shots only. In our experiments, we found that identifying GUI widgets in screen shots and using this information to guide random testing achieved a significantly higher branch coverage in 18 of 20 applications, with an average increase of 42.5% when compared to conventional random testing.

Graphical user interface (GUI) is an important feature of current software, and testing for GUI is considered as the most efficient method to guarantee the quality of GUI. It is consensus that in industry and academia automated GUI testing approach and tool have been a great help in reducing time and cost in testing, especially regression testing. However, almost all existing automated GUI testing approaches are code intrusive, and strongly dependent on platform or system environment, which leads to the lack of general applicability of GUI testing tools. And most of these tools can only be used in specific software or software platforms. To deal with the challenge, this paper proposes a novel automated GUI testing based on image recognition, which is a non-intrusive and general GUI testing approach. A platform called Auto GUI Testing Platform (AGTP) based on our approach is developed. And a case study is proposed to demonstrate the validity and practicability of AGTP.

Software testing has become an important phase in software applications’ lifecycle. Graphical User Interface (GUI) components can be found in a large number of desktops and web applications and also in a wide variety of mobile devices. In the last years GUIs have become more and more complex and interactive, their testing process requiring an interaction with the GUI components, mainly by generating mouse, keyboard and touch events. Given their increased importance, GUIs verification for correctness contributes to the establishment of the correct functionality of the corresponding software application. The current research on GUI testing methodologies primarily focuses on automated testing. This paper presents EvoGUITest, a novel automated GUI testing framework based on evolutionary algorithms which tests the GUI independently from the application code itself. The framework is designed for testing GUIs of web applications. Results have been compared, based on specific metrics, with others existing frameworks.

The paper provides the design of a distributed test system for testing distributed GUI (Graphical User Interface) applications. The distributed GUI application that is considered allows multiple operators or users to interact with a distributed system in parallel. The distributed GUI application allows the use of existing GUI test frameworks for local test execution and offers additional services of coordination and synchronization among the deployed GUI test frameworks, test verdict arbitration, and debugging support, all of which are needed to deal with the concurrent nature of the system under test. In addition, the paper offers a test specification method for test cases that is based on the semantic model of a concurrent event flow graph. This method allows the automatic test case execution on top of the suggested distributed test system. The overall approach allows the extension of testing of single GUI application systems towards testing of distributed GUI application systems by preserving investments in test automation for single GUI testing.

In recent times, software industry has seen the immense expansion in its popularity. Graphical user interface (GUI) is of the most important and ground breaking factor behind the exponential growth of acceptance of software systems. Beside this popularity, software organizations are trying best to optimize the quality of delivered products. In software development life cycle, software testing is one major phase having focus towards improved software quality. Increased attention towards GUI in development helped in materialization of GUI testing as a vital branch of software testing. Manual efforts to test GUI have revealed a lot of problems, which, ultimately led towards automation of GUI testing. This paper presents a novel approach towards automation of GUI test coverage analysis based on an evolutionary algorithm, Particle Swarm Optimization (PSO). Another objective that has been tried to achieve is keeping number of test cases at minimum side.

Graphical user interfaces (GUIs) for today's applications are extremely large. Moreover, they provide many degrees of freedom to the end-user, thus allowing the user to perform a very large number of event sequences on the GUI. The large sizes and degrees of freedom create severe problems for GUI quality assurance, including GUI testing. In this paper, we leverage methods and measures from network analysis to analyze and study GUIs, with the goal of aiding GUI testing activities. We apply these methods and measures on the event-flow graph model of GUIs. Results of a case study show that "network centrality measures" are able to identify the most important events in the GUI as well as the most important sequences of events. These events and sequences are good candidates for test prioritization. In addition, the "betweenness clustering" method is able to partition the GUI into regions that can be tested separately.

This paper introduces an automated grader for Java programs called GUI_Grader that allows students a degree of flexibility in graphical user interface (GUI) design. GUI_Grader allows students to build multi-window Java applications, choose among alternative GUI components, and decide how to order, position and label components. This enables students to practice some aspects of designing their own GUI applications while still providing automated grading `based on a single test plan. The data-driven approach helps to maintain consistency between test plans and program specifications. Testing GUI_Grader on Java assignments from both a first-year and an upper-year course confirms the feasibility of the approach.

Short Paper; Topics: Verification/Validation, Planning Graphical User Interfaces (GUIs) are very popular and common in computer-based systems. Testing GUIs is, on the other hand, a difficult and challenging task for many reasons: First, the input space possesses a great, potentially indefinite number of combinations of inputs and events as system outputs and external events may interact with these inputs. Second, even simple GUIs possess an enormous number of states which are also due to interact with the inputs. Last but not least, many complex dependencies may hold between the states of the GUI system, and/or between its states and -inputs.Nevertheless, most Human-Computer-Interfaces (HCI) will be designed and implemented via GUI nowadays. There exist a vast amount of research work for specification of HCI, there has been, however, little well-known systematic study in this field, resulting an effective testing strategy which is not only easy to apply, but also scalable in sense of stepwise increasing the number of test cases, thus also increasing the test costs in accordance with the test budget of the project.

Graphical User Interfaces (GUIs) are substantial parts of today's applications, no matter whether these run on tablets, smartphones or desktop platforms. Since the GUI is often the only component that humans interact with, it demands for thorough testing to ensure an efficient and satisfactory user experience. Being the glue between almost all of an application's components, GUIs also lend themselves for system level testing. However, GUI testing is inherently difficult and often involves great manual labor, even with modern tools which promise automation. This paper introduces a Java library called GUITest, which allows to generate fully automated GUI robustness tests for complex applications, without the need to manually generate models or input sequences. We will explain how it operates and present first results on its applicability and effectivity during a test involving Microsoft Word.

The purpose of Graphical User Interfaces (GUI) testing is to diagnose and expose faults in planning time. It is difficult because the input space of GUI is extremely large due to different permutations of inputs and events. To test GUI needs to run a lot of test cases. Neural networks (NN) were explored to reduce test cases to expose new faults. The main idea is as follows. Firstly, NN was trained by subset of test cases that had executed and their test results. Trained NN could recognize fault patterns that had been exposed. Secondly, from the test suite that hadn't been executed, trained NN was used to select test cases that don't belong to the fault patterns. The test cases selected were more likely to expose new faults in GUI. By the method new faults could be exposed by executing fewer test cases. The experimental results show that the strategy is effective.

Automated testing in web applications is becoming increasingly popular due to its cost-effectiveness, efficiency, and repeatability. Ensuring the quality of such software is vital, especially in scenarios where business requirements are uncertain and volatile. In this paper, we propose Cytestion, a tool that automatically explores a web application’s Graphical User Interface (GUI) to detect faults that cause visible failures, such as crashes, errors, and unexpected behaviors. For that, it applies a scriptless and progressive approach. To work as an oracle, Cytestion evaluates the output of each test case execution by considering the HTTP status of server requests, default and customized failure GUI messages, and console failure messages. We evaluated the effectiveness and cost of our approach in two empirical studies. The first study works with injected faults in open-source projects, while the second is a case study on eight industrial projects. Cytestion detected several faults in both studies and outperformed a state-of-the-art GUI testing tool.