Distributed GUI Test Automation

This paper introduces an automated grader for Java programs called GUI_Grader that allows students a degree of flexibility in graphical user interface (GUI) design. GUI_Grader allows students to build multi-window Java applications, choose among alternative GUI components, and decide how to order, position and label components. This enables students to practice some aspects of designing their own GUI applications while still providing automated grading `based on a single test plan. The data-driven approach helps to maintain consistency between test plans and program specifications. Testing GUI_Grader on Java assignments from both a first-year and an upper-year course confirms the feasibility of the approach.

Although there have been major achievements in verified software, work on verifying graphical user interfaces (GUI) applications is underdeveloped relative to their ubiquity and societal importance. In this paper, we present a library for the development of verified, state-dependent GUI applications in the dependently typed programming language Agda. The library uses Agda’s expressive type system to ensure that the GUI, its controller, and the underlying model are all consistent, significantly reducing the scope for GUI-related bugs. We provide a way to specify and prove correctness properties of GUI applications in terms of user interactions and state transitions. Critically, GUI applications and correctness properties are not restricted to finite state machines and may involve the execution of arbitrary interactive programs. Additionally, the library connects to a standard, imperative GUI framework, enabling the development of native GUI applications with expected features, such as concurrency. We present applications of our library to building GUI applications to manage healthcare processes. The correctness properties we consider are the following: (1) That a state can only be reached by passing through a particular intermediate state, for example, that a particular treatment can only be reached after having conducted an X-Ray. (2) That one eventually reaches a particular state, for example, that one eventually decides on a treatment. The specification of such properties is defined in terms of a GUI application simulator, which simulates all possible sequences of interactions carried out by the user.

Modeling and simulation (M&S) is increasingly used in drug development to characterize pharmacokinetic-pharmacodynamic (PKPD) relationships and support various efforts such as target feasibility assessment, molecule selection, human PK projection, and preclinical and clinical dose and schedule determination. While model development typically require mathematical modeling expertise, model exploration and simulations could in many cases be performed by scientists in various disciplines to support the design, analysis and interpretation of experimental studies. To this end, we have developed a versatile graphical user interface (GUI) application to enable easy use of any model constructed in SimBiology® to execute various common PKPD analyses. The MATLAB®-based GUI application, called gPKPDSim, has a single screen interface and provides functionalities including simulation, data fitting (parameter estimation), population simulation (exploring the impact of parameter variability on the outputs of interest), and non-compartmental PK analysis. Further, gPKPDSim is a user-friendly tool with capabilities including interactive visualization, exporting of results and generation of presentation-ready figures. gPKPDSim was designed primarily for use in preclinical and translational drug development, although broader applications exist. gPKPDSim is a MATLAB®-based open-source application and is publicly available to download from MATLAB® Central™. We illustrate the use and features of gPKPDSim using multiple PKPD models to demonstrate the wide applications of this tool in pharmaceutical sciences. Overall, gPKPDSim provides an integrated, multi-purpose user-friendly GUI application to enable efficient use of PKPD models by scientists from various disciplines, regardless of their modeling expertise.

<div><p class="Els-history-head">The development of rocketry and rocket payload technology is very rapid in the industrial era 4.0. Rockets are usually used as dynamic sensors or carriers for specific missions for sensing and retrieving data from space for meteorological, military, etc. This article aims to design, realize and test a Rocket Payload Monitoring System Based on an Inertial Measurement Unit (IMU) Sensor to monitor the condition of the payload movement behavior. Rocket Payload is a substance carried in a Payload Test Rocket (RUM) in the form of a cylindrical payload. This research focuses on designing a Graphical User Interface (GUI) application to display data in real-time. The type of research used is Research and Development (R&D) with a prototyping development model. The results obtained in the form of Rocket Payload Design and GUI design can be concluded that the GUI application can visualize all sensor and camera data in real-time.</p></div>

Graphical user interfaces (GUIs) are ubiquitous in real-world software and a notorious source of bugs that are difficult to catch through software testing. Model checking has been used to prove the absence of certain kinds of bugs, but model checking works on an abstract model of the GUI application, which might be inconsistent with its implementation. We present a library for developing directly verified, state-dependent GUI applications in the dependently typed programming language Agda. In the library, the type of a GUI's controller depends on a specification of the GUI itself, statically enforcing consistency between them. Arbitrary properties can be defined and proved in terms of user interactions and state transitions. Our library connects to a custom-built Haskell back-end for declarative vector-based GUI elements. Compared to an earlier version of our library built on an existing imperative GUI framework, the more declarative back-end supports simpler definitions and proofs. As a practical application of our library to a safety-critical domain, we present a case study developed in cooperation with the Medical University of Vienna. The case study implements a healthcare process for prescribing anticoagulants, which is highly error-prone when followed manually. Our implementation generates GUIs from an abstract description of a data-aware business process, making our approach easy to reuse and adapt to other safety-critical processes. We prove medically relevant safety properties about the executable GUI application, such as that given certain inputs, certain states must or must not be reached.

Software testing in general and graphical user interface (GUI) testing in particular is one of the major challenges in the lifecycle of any software system. GUI testing is inherently more difficult than the traditional and command-line interface testing. Some of the factors that make GUI testing different from the traditional software testing and significantly more difficult are: a large number of objects, different look and feel of objects, many parameters associated with each object, progressive disclosure, complex inputs from multiple sources, and graphical outputs. The existing testing techniques for the creation and management of test suites need to be adapted/enhanced for GUIs, and new testing techniques are desired to make the creation and management of test suites more efficient and effective. In this article, a methodology is proposed to create test suites for a GUI. The proposed methodology organizes the testing activity into various levels. The tests created at a particular level can be reused at higher levels. This methodology extends the notion of modularity and reusability to the testing phase. The organization and management of the created test suites resembles closely to the structure of the GUI under test.

The practitioner interested in reducing software verification effort may found herself lost in the many alternative definitions of Graphical User Interface (GUI) testing that exist and their relation to the notion of system testing. One result of these many definitions is that one may end up testing the same parts of the Software Under Test (SUT), specifically the application logic, twice. To clarify two important testing activities and avoid duplicate testing effort, this paper empirically evaluates to what extent GUI tests exercise the application logic of the software under test (and not only the GUI code). Experimental results show that GUI tests do not necessarily entirely exercise application logic functionality, at least not as much as system tests directly interacting with application logic code.

Verifying the functional behavior of graphical user interface (GUI) applications is essential for reducing post-release issues. In practice, a developer/tester performs this verification by executing a sequence of GUI actions and then witnessing the expected behavior on the GUI screen. An automated witness generator facilitates the verification process. However, creating an unambiguous and monitorable specification for the correct behavior and then generating the correct GUI actions to trigger that behavior is challenging. In this study, we propose FARLEAD2, an automated witness generator that uses unambiguous, monitorable, and easy-to-read staged test scenarios (STSs) to specify expected behavior. FARLEAD2 maximizes its effectiveness and performance using generalized experienced replay (GER) to exploit the experience gathered from previously witnessed scenarios on new, unwitnessed test scenarios. To the best of our knowledge, STS and GER are novel improvements to GUI testing. Our evaluation of Android GUI applications shows that FARLEAD2 effectively generates a witness 95.7 times out of 100 and does it in 520 seconds, on average, indicating that FARLEAD2 is approximately 65 percent faster and 6.3 percent more effective than its best predecessor.

Graphical User Interfaces (GUIs) are amongst the most common user interfaces, enabling interactions with applications through mouse movements and key presses. Tools for automated testing of programs through their GUI exist, however they usually rely on operating system or framework specific knowledge to interact with an application. Due to frequent operating system updates, which can remove required information, and a large variety of different GUI frameworks using unique underlying data structures, such tools rapidly become obsolete, Consequently, for an automated GUI test generation tool, supporting many frameworks and operating systems is impractical. We propose a technique for improving GUI testing by automatically identifying GUI widgets in screen shots using machine learning techniques. As training data, we generate randomized GUIs to automatically extract widget information. The resulting model provides guidance to GUI testing tools in environments not currently supported by deriving GUI widget information from screen shots only. In our experiments, we found that identifying GUI widgets in screen shots and using this information to guide random testing achieved a significantly higher branch coverage in 18 of 20 applications, with an average increase of 42.5% when compared to conventional random testing.

Graphical User Interface (GUI) has become one of the most significant parts of mobile applications (apps). It is a direct bridge between mobile apps and end users, which directly affects the end user’s experience. Neglecting GUI quality can undermine the value and effectiveness of the entire mobile app solution. Significant research efforts have been devoted to GUI testing, one effective method to ensure mobile app quality. By conducting rigorous GUI testing, developers can ensure that the visual and interactive elements of the mobile apps not only meet functional requirements but also provide a seamless and user-friendly experience. However, traditional solutions, relying on the source code or layout files, have met challenges in both effectiveness and efficiency due to the gap between what is obtained and what app GUI actually presents. Vision-based mobile app GUI testing approaches emerged with the development of computer vision technologies and have achieved promising progress. In this survey article, we provide a comprehensive investigation of the state-of-the-art techniques on 271 articles, among which 92 are vision-based studies. This survey covers different topics of GUI testing, like GUI test generation, GUI test record & replay, GUI testing framework, and so on. In particular, we highlight the emerging role of vision-based techniques and analyze how they reshape traditional approaches to mobile app GUI testing. Based on the investigation of existing studies, we outline the challenges and opportunities of (vision-based) mobile app GUI testing and propose promising research directions with the combination of emerging techniques.

High Performance Computing (HPC) applications commonly use Message Passing Interface (MPI) to scale complex workloads on distributed memory architectures, while Graphical User Interface (GUI) is the primary basis for pre-processing and post-processing. Containerization enables portability and reproducibility for applications. Docker is a widely used containerization platform but cannot be easily adopted in secure HPC environments due to the need for root privileges to run Docker containers and the ensuing security issues, as well as the lack of native support for Message Passing Interface (MPI) and HPC workload managers. In addition to offering portability and reproducibility to scientific computing, Singularity and Sarus are container runtimes that enable containers to run on HPC with user credentials by removing security concerns and allowing reuse of Docker containers without any changes. This work presents approaches to creating and running MPI and GUI applications in an isolated container environment on HPC in the cloud. We provide performance evaluation of application containers transformed from Docker to Singularity and Sarus in the context of native on HPC in the cloud.

The R environment provides a natural platform for developing new statistical methods due to the mathematical expressiveness of the language, the large number of existing libraries, and the active developer community. One drawback to R, however, is the learning curve; programming is a deterrent to non-technical users, who typically prefer graphical user interfaces (GUIs) to command line environments. Thus, while statisticians develop new methods in R, practitioners are often behind in terms of the statistical techniques they use as they rely on GUI applications. Meta-analysis is an instructive example; cutting-edge meta-analysis methods are often ignored by the overwhelming majority of practitioners, in part because they have no easy way of applying them. This paper proposes a strategy to close the gap between the statistical state-of-the-science and what is applied in practice. We present open-source meta-analysis software that uses R as the underlying statistical engine, and Python for the GUI. We present a framework that allows methodologists to implement new methods in R that are then automatically integrated into the GUI for use by end-users, so long as the programmer conforms to our interface. Such an approach allows an intuitive interface for non-technical users while leveraging the latest advanced statistical methods implemented by methodologists.

Graphical user interface (GUI) is an important feature of current software, and testing for GUI is considered as the most efficient method to guarantee the quality of GUI. It is consensus that in industry and academia automated GUI testing approach and tool have been a great help in reducing time and cost in testing, especially regression testing. However, almost all existing automated GUI testing approaches are code intrusive, and strongly dependent on platform or system environment, which leads to the lack of general applicability of GUI testing tools. And most of these tools can only be used in specific software or software platforms. To deal with the challenge, this paper proposes a novel automated GUI testing based on image recognition, which is a non-intrusive and general GUI testing approach. A platform called Auto GUI Testing Platform (AGTP) based on our approach is developed. And a case study is proposed to demonstrate the validity and practicability of AGTP.

Graphical user interfaces (GUIs) for today's applications are extremely large. Moreover, they provide many degrees of freedom to the end-user, thus allowing the user to perform a very large number of event sequences on the GUI. The large sizes and degrees of freedom create severe problems for GUI quality assurance, including GUI testing. In this paper, we leverage methods and measures from network analysis to analyze and study GUIs, with the goal of aiding GUI testing activities. We apply these methods and measures on the event-flow graph model of GUIs. Results of a case study show that "network centrality measures" are able to identify the most important events in the GUI as well as the most important sequences of events. These events and sequences are good candidates for test prioritization. In addition, the "betweenness clustering" method is able to partition the GUI into regions that can be tested separately.

This position paper presents a synopsis of the imperative function artificial intelligence (AI) has partaken in software engineering (SE) as well as in software testing. In addition, the paper discusses how graphical user interface (GUI), and event driven software testing can derive benefits from the use of AI techniques. Artificial intelligence has significantly aided the process of the automation of different software process. The employment of AI in software testing is not novel, having played a crucial role in the automation of software testing since its innovation. The usage of AI techniques not only reduces the cost but it also guarantees better quality as well as thorough testing. GUI Testing can be considered as the most challenging area of software testing. Although the results are quite preliminary, but the application of different AI techniques for GUI testing has proven to produce ideal results. Nevertheless, the application of AI techniques in GUI testing, in comparison with software testing, which has procured much assistance by venturing with AI, leaves much to be desired.