Comparison Analysis of Android GUI Testing Frameworks by Using an Experimental Study

Graphical User Interfaces (GUIs) are amongst the most common user interfaces, enabling interactions with applications through mouse movements and key presses. Tools for automated testing of programs through their GUI exist, however they usually rely on operating system or framework specific knowledge to interact with an application. Due to frequent operating system updates, which can remove required information, and a large variety of different GUI frameworks using unique underlying data structures, such tools rapidly become obsolete, Consequently, for an automated GUI test generation tool, supporting many frameworks and operating systems is impractical. We propose a technique for improving GUI testing by automatically identifying GUI widgets in screen shots using machine learning techniques. As training data, we generate randomized GUIs to automatically extract widget information. The resulting model provides guidance to GUI testing tools in environments not currently supported by deriving GUI widget information from screen shots only. In our experiments, we found that identifying GUI widgets in screen shots and using this information to guide random testing achieved a significantly higher branch coverage in 18 of 20 applications, with an average increase of 42.5% when compared to conventional random testing.

Graphical User Interface (GUI) has become one of the most significant parts of mobile applications (apps). It is a direct bridge between mobile apps and end users, which directly affects the end user’s experience. Neglecting GUI quality can undermine the value and effectiveness of the entire mobile app solution. Significant research efforts have been devoted to GUI testing, one effective method to ensure mobile app quality. By conducting rigorous GUI testing, developers can ensure that the visual and interactive elements of the mobile apps not only meet functional requirements but also provide a seamless and user-friendly experience. However, traditional solutions, relying on the source code or layout files, have met challenges in both effectiveness and efficiency due to the gap between what is obtained and what app GUI actually presents. Vision-based mobile app GUI testing approaches emerged with the development of computer vision technologies and have achieved promising progress. In this survey article, we provide a comprehensive investigation of the state-of-the-art techniques on 271 articles, among which 92 are vision-based studies. This survey covers different topics of GUI testing, like GUI test generation, GUI test record & replay, GUI testing framework, and so on. In particular, we highlight the emerging role of vision-based techniques and analyze how they reshape traditional approaches to mobile app GUI testing. Based on the investigation of existing studies, we outline the challenges and opportunities of (vision-based) mobile app GUI testing and propose promising research directions with the combination of emerging techniques.

In software testing process, test data generation represents an important step for high quality software, even for mobile devices. As proposed in previous works, a potential source for random data generation is represented by the UI layout files that are used for almost all mobile platforms (Android, iOS, Windows Phone/Mobile). This paper continues the previous work and presents a test data generation system based on Android layout files. The test data generator uses DSL files as input and generates test data that conform to several testing principles. The generated test data could be stored in XML files or any format required by the testing frameworks.

Software testing in general and graphical user interface (GUI) testing in particular is one of the major challenges in the lifecycle of any software system. GUI testing is inherently more difficult than the traditional and command-line interface testing. Some of the factors that make GUI testing different from the traditional software testing and significantly more difficult are: a large number of objects, different look and feel of objects, many parameters associated with each object, progressive disclosure, complex inputs from multiple sources, and graphical outputs. The existing testing techniques for the creation and management of test suites need to be adapted/enhanced for GUIs, and new testing techniques are desired to make the creation and management of test suites more efficient and effective. In this article, a methodology is proposed to create test suites for a GUI. The proposed methodology organizes the testing activity into various levels. The tests created at a particular level can be reused at higher levels. This methodology extends the notion of modularity and reusability to the testing phase. The organization and management of the created test suites resembles closely to the structure of the GUI under test.

The practitioner interested in reducing software verification effort may found herself lost in the many alternative definitions of Graphical User Interface (GUI) testing that exist and their relation to the notion of system testing. One result of these many definitions is that one may end up testing the same parts of the Software Under Test (SUT), specifically the application logic, twice. To clarify two important testing activities and avoid duplicate testing effort, this paper empirically evaluates to what extent GUI tests exercise the application logic of the software under test (and not only the GUI code). Experimental results show that GUI tests do not necessarily entirely exercise application logic functionality, at least not as much as system tests directly interacting with application logic code.

Graphical user interface (GUI) is an important feature of current software, and testing for GUI is considered as the most efficient method to guarantee the quality of GUI. It is consensus that in industry and academia automated GUI testing approach and tool have been a great help in reducing time and cost in testing, especially regression testing. However, almost all existing automated GUI testing approaches are code intrusive, and strongly dependent on platform or system environment, which leads to the lack of general applicability of GUI testing tools. And most of these tools can only be used in specific software or software platforms. To deal with the challenge, this paper proposes a novel automated GUI testing based on image recognition, which is a non-intrusive and general GUI testing approach. A platform called Auto GUI Testing Platform (AGTP) based on our approach is developed. And a case study is proposed to demonstrate the validity and practicability of AGTP.

The paper provides the design of a distributed test system for testing distributed GUI (Graphical User Interface) applications. The distributed GUI application that is considered allows multiple operators or users to interact with a distributed system in parallel. The distributed GUI application allows the use of existing GUI test frameworks for local test execution and offers additional services of coordination and synchronization among the deployed GUI test frameworks, test verdict arbitration, and debugging support, all of which are needed to deal with the concurrent nature of the system under test. In addition, the paper offers a test specification method for test cases that is based on the semantic model of a concurrent event flow graph. This method allows the automatic test case execution on top of the suggested distributed test system. The overall approach allows the extension of testing of single GUI application systems towards testing of distributed GUI application systems by preserving investments in test automation for single GUI testing.

In this paper we investigate the effect of test case length on the performance of test suites for automated graphical user interface (GUI) testing. Research has been done on the effect of test length for traditional white-box testing, however, these findings do not necessarily extend to GUI testing. We run experiments on five subject applications using a custom-built dynamic test generator. We measure the effect that test length has on statement, branch, and event-pair coverage, as well as the fault-finding abilities for each test suite. Our findings indicate that longer test suites perform better than shorter test suites in terms of coverage, fault-finding, and cost. However, the effect eventually diminishes with greater increase in length. As well, the results vary depending on the application under test.

This position paper presents a synopsis of the imperative function artificial intelligence (AI) has partaken in software engineering (SE) as well as in software testing. In addition, the paper discusses how graphical user interface (GUI), and event driven software testing can derive benefits from the use of AI techniques. Artificial intelligence has significantly aided the process of the automation of different software process. The employment of AI in software testing is not novel, having played a crucial role in the automation of software testing since its innovation. The usage of AI techniques not only reduces the cost but it also guarantees better quality as well as thorough testing. GUI Testing can be considered as the most challenging area of software testing. Although the results are quite preliminary, but the application of different AI techniques for GUI testing has proven to produce ideal results. Nevertheless, the application of AI techniques in GUI testing, in comparison with software testing, which has procured much assistance by venturing with AI, leaves much to be desired.

Software testing has become an important phase in software applications’ lifecycle. Graphical User Interface (GUI) components can be found in a large number of desktops and web applications and also in a wide variety of mobile devices. In the last years GUIs have become more and more complex and interactive, their testing process requiring an interaction with the GUI components, mainly by generating mouse, keyboard and touch events. Given their increased importance, GUIs verification for correctness contributes to the establishment of the correct functionality of the corresponding software application. The current research on GUI testing methodologies primarily focuses on automated testing. This paper presents EvoGUITest, a novel automated GUI testing framework based on evolutionary algorithms which tests the GUI independently from the application code itself. The framework is designed for testing GUIs of web applications. Results have been compared, based on specific metrics, with others existing frameworks.

Graphical user interfaces (GUIs) for today's applications are extremely large. Moreover, they provide many degrees of freedom to the end-user, thus allowing the user to perform a very large number of event sequences on the GUI. The large sizes and degrees of freedom create severe problems for GUI quality assurance, including GUI testing. In this paper, we leverage methods and measures from network analysis to analyze and study GUIs, with the goal of aiding GUI testing activities. We apply these methods and measures on the event-flow graph model of GUIs. Results of a case study show that "network centrality measures" are able to identify the most important events in the GUI as well as the most important sequences of events. These events and sequences are good candidates for test prioritization. In addition, the "betweenness clustering" method is able to partition the GUI into regions that can be tested separately.

This paper introduces an automated grader for Java programs called GUI_Grader that allows students a degree of flexibility in graphical user interface (GUI) design. GUI_Grader allows students to build multi-window Java applications, choose among alternative GUI components, and decide how to order, position and label components. This enables students to practice some aspects of designing their own GUI applications while still providing automated grading `based on a single test plan. The data-driven approach helps to maintain consistency between test plans and program specifications. Testing GUI_Grader on Java assignments from both a first-year and an upper-year course confirms the feasibility of the approach.

The current increase in smartphone users has caused various mobile device operating system companies to compete with each other to create a mobile device operating system that is suitable and acceptable to the public. Competition among mobile operating system market shares can be analyzed using the Lotka-Volterra model. This study aims to reconstruct the Lotka-Volterra model for the cellular operating system market share in Indonesia. In addition, a stability analysis was carried out, which aims to determine the stability of the competitive model for the market share of the operating system in Indonesia. The results of the study show that a competitive Lotka-Volterra model can be built on the Android and iOS operating system market share in Indonesia. In this model, there are four equilibrium points, one of which is unstable, and the other three equilibrium points are conditionally stable.

Short Paper; Topics: Verification/Validation, Planning Graphical User Interfaces (GUIs) are very popular and common in computer-based systems. Testing GUIs is, on the other hand, a difficult and challenging task for many reasons: First, the input space possesses a great, potentially indefinite number of combinations of inputs and events as system outputs and external events may interact with these inputs. Second, even simple GUIs possess an enormous number of states which are also due to interact with the inputs. Last but not least, many complex dependencies may hold between the states of the GUI system, and/or between its states and -inputs.Nevertheless, most Human-Computer-Interfaces (HCI) will be designed and implemented via GUI nowadays. There exist a vast amount of research work for specification of HCI, there has been, however, little well-known systematic study in this field, resulting an effective testing strategy which is not only easy to apply, but also scalable in sense of stepwise increasing the number of test cases, thus also increasing the test costs in accordance with the test budget of the project.

Graphical User Interfaces (GUIs) are substantial parts of today's applications, no matter whether these run on tablets, smartphones or desktop platforms. Since the GUI is often the only component that humans interact with, it demands for thorough testing to ensure an efficient and satisfactory user experience. Being the glue between almost all of an application's components, GUIs also lend themselves for system level testing. However, GUI testing is inherently difficult and often involves great manual labor, even with modern tools which promise automation. This paper introduces a Java library called GUITest, which allows to generate fully automated GUI robustness tests for complex applications, without the need to manually generate models or input sequences. We will explain how it operates and present first results on its applicability and effectivity during a test involving Microsoft Word.