Performance Evaluation of Graphical User Interfaces in Java and C#

Software testing has become an important phase in software applications’ lifecycle. Graphical User Interface (GUI) components can be found in a large number of desktops and web applications and also in a wide variety of mobile devices. In the last years GUIs have become more and more complex and interactive, their testing process requiring an interaction with the GUI components, mainly by generating mouse, keyboard and touch events. Given their increased importance, GUIs verification for correctness contributes to the establishment of the correct functionality of the corresponding software application. The current research on GUI testing methodologies primarily focuses on automated testing. This paper presents EvoGUITest, a novel automated GUI testing framework based on evolutionary algorithms which tests the GUI independently from the application code itself. The framework is designed for testing GUIs of web applications. Results have been compared, based on specific metrics, with others existing frameworks.

The increasing expectations of users for ever more functional and impressive Graphical User Interfaces (GUIs) have instigated changes to the way in which programming languages are taught. The rise in popularity of the subject matter has caused problems for lecturers with regards to time and costs. The numbers of student solutions that need to be marked in short periods of time are barely feasible for one person alone or even a marking team to manage; this has been a major influence in the development of marking systems. No system exists for marking GUI programs, indeed few exist for marking any programs. This paper presents the concepts behind an introspective testing and marking tool for GUIs. The hierarchical relationship between interface components and the use of an introspective approach allows us to retrieve data from the GUI components themselves, rather than through relative estimations. This not only allows for dynamic testing of the student's solutions but also aesthetic testing measures.

Application development of a large number of graphical user interface (GUI) screens and reports requires a labor‐intensive process. In the typical scenario, a GUI developer drags each component, such as the textfield, label, and error messages, on each of the new GUI screens or reports. They then set the component properties and perhaps write any validation routines for input data. Using the proposed Observation Panel Bean (OPB), the developer drags only one component per field, and the OPB dynamically builds the GUI components with consistent properties and links the observation field to the needed validation routine. With little development effort these fields are simply selected from the data dictionary and placed on the GUI. The proposed OPB uses Fowler's observation pattern as a high‐level abstraction representing attributes observed about information in the application domain. The panel collaborates with the observation pattern to set the label, error message, and properties of the various components. Yoder's validator pattern collaborates with the observation pattern and the data dictionary to validate the observations. The proposed OPB decreases GUI development time, adds adaptability to GUI design, and increases consistency in GUI component presentation. Copyright © 2002 John Wiley & Sons, Ltd.

All most all of the bugs related to graphical user interface (GUI) module of applications and are described in terms of events associated with GUI components. In this paper, a bug mining model for discovering duplicate and similar GUI bugs is presented and approach for detecting the similar and duplicate GUI bugs is described. Resolution of similar and duplicate bugs are almost identical, so if similar and duplicates are identified it will optimize the time for fixing reported GUI bugs and it can also help in achieving the faster development. A GUI bug can be transformed into a sequence of events, components and expected implementation requirements for each GUI event. This transformation is used in this paper to discover the similar and duplicate GUI bugs. First all the GUI bugs are transformed into events, components and requirements sequence, then these sequences are pair wise matched and common subsequence is generated which will indicate the similarity for the GUI bugs.

VERITAS (Visualization Environment Research In The Applied Sciences) involves the design and development of tools for the creation of user environments for scientific computing systems. The initial goal has been to provide a tool-based environment in which a scientist can easily and rapidly construct, with no programming effort, a powerful customized graphical user interface to an existing scientific application. The ultimate goal is to support in a similar interactive, high-level way the development of the entire scientific application system by the scientist, to allow a tighter integration of the application model and the graphical interface. The project involves research and development in visualization and interactive data presentation techniques, knowledge representation and management systems, and high-level specification and programming languages.

Graphical User Interface (GUI) provides a visual bridge between a software application and end users, through which they can interact with each other. With the upgrading of mobile devices and the development of aesthetics, the visual effects of the GUI are more and more attracting, and users pay more attention to the accessibility and usability of applications. However, such GUI complexity posts a great challenge to the GUI implementation. According to our pilot study of crowdtesting bug reports, display issues such as text overlap, component occlusion, missing image always occur during GUI rendering on different devices due to the software or hardware compatibility. They negatively influence the app usability, resulting in poor user experience. To detect these issues, we propose a fully automated approach, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Nighthawk</monospace> , based on deep learning for modelling visual information of the GUI screenshot. <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Nighthawk</monospace> can detect GUIs with display issues and also locate the detailed region of the issue in the given GUI for guiding developers to fix the bug. At the same time, training the model needs a large amount of labeled buggy screenshots, which requires considerable manual effort to prepare them. We therefore propose a heuristic-based training data auto-generation method to automatically generate the labeled training data. The evaluation demonstrates that our <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Nighthawk</monospace> can achieve average 0.84 precision and 0.84 recall in detecting UI display issues, average 0.59 AP and 0.60 AR in localizing these issues. We also evaluate <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Nighthawk</monospace> with popular Android apps on Google Play and F-Droid, and successfully uncover 151 previously-undetected UI display issues with 75 of them being confirmed or fixed so far.

This paper introduces an automated grader for Java programs called GUI_Grader that allows students a degree of flexibility in graphical user interface (GUI) design. GUI_Grader allows students to build multi-window Java applications, choose among alternative GUI components, and decide how to order, position and label components. This enables students to practice some aspects of designing their own GUI applications while still providing automated grading `based on a single test plan. The data-driven approach helps to maintain consistency between test plans and program specifications. Testing GUI_Grader on Java assignments from both a first-year and an upper-year course confirms the feasibility of the approach.

Currently, the rapid growth of mobile apps requires automated testing technology to ensure their quality. Automated testing of mobile apps is usually closely related to the recognition and judgment of their graphical user interface (GUI), but there usually are numerous isomorphic GUIs with different styles and contents, and similar structure and function in mobile apps. In automatic testing, isomorphic GUI is easy to cause the issue of state space explosion, which leads to low efficiency or failure of testing. In view of the limitations of traditional automatic recognition of isomorphic GUI, this paper presents a GUI similarity judgment approach based on visual feature information. Firstly, the GUI component elements are identified by object detection, and then the GUI skeleton is abstracted. Secondly, the visual features of the GUI skeleton are extracted by a convolutional autoencoder. Finally, the isomorphic GUI judgment is completed by comparing the similarity of GUI visual features. The experimental results show that the proposed approach can effectively shield the influence of GUI style and content, complete the isomorphic GUI recognition more accurately and optimize the efficiency of automated mobile app testing.

Visual and aural are two most important channels of information processing. While most of the interaction with computers have been designed around the visual channel, there are circumstances where voice based man-machine interaction becomes preferable, and in some cases, necessary, given that voice based interaction comes naturally to humans and can be used by illiterate people easily.

Graphical User Interface (GUI) provides a visual bridge between a software\napplication and end users, through which they can interact with each other.\nWith the development of technology and aesthetics, the visual effects of the\nGUI are more and more attracting. However, such GUI complexity posts a great\nchallenge to the GUI implementation. According to our pilot study of\ncrowdtesting bug reports, display issues such as text overlap, blurred screen,\nmissing image always occur during GUI rendering on different devices due to the\nsoftware or hardware compatibility. They negatively influence the app\nusability, resulting in poor user experience. To detect these issues, we\npropose a novel approach, OwlEye, based on deep learning for modelling visual\ninformation of the GUI screenshot. Therefore, OwlEye can detect GUIs with\ndisplay issues and also locate the detailed region of the issue in the given\nGUI for guiding developers to fix the bug. We manually construct a large-scale\nlabelled dataset with 4,470 GUI screenshots with UI display issues and develop\na heuristics-based data augmentation method for boosting the performance of our\nOwlEye. The evaluation demonstrates that our OwlEye can achieve 85% precision\nand 84% recall in detecting UI display issues, and 90% accuracy in localizing\nthese issues. We also evaluate OwlEye with popular Android apps on Google Play\nand F-droid, and successfully uncover 57 previously-undetected UI display\nissues with 26 of them being confirmed or fixed so far.\n

Graphical User Interfaces (GUIs) represent a key part for building interactive applications, and they act as a direct interface between users and software. The complexity of the GUIs increases in relation to the number of users interacting with software and performing multiple complex tasks, resulting in software that are mazes of functionalities. The impact of such a GUI system is adverse on the software quality and may result in non-robust software. To measure software quality, one has to measure attributes such as usability, efficiency, and learnability. There is no general approach for measuring software validity subjectively in a non-invasive way, i.e., without directly asking the users to evaluate directly the software through predefined use cases or through questionnaires. In this paper, we investigate this issue by detecting the presence of potential errors in GUIs by using the users gaze information collected during user interactions. Users gaze is analyzed to generate thirteen unique features based on statistical moments based on the time spent on each control, gaze path, and transitions between controls on the different GUI components present on the screen. The performance is evaluated with five state of the art classifiers. The results support the conclusion that incongruent GUI states can be reliably detected at the single trial level through gaze-based features with an accuracy of 76% (one-versus-rest classification) and 73.66 (one-versus-one classification) using a Gaussian Naive Bayes classifier. These results suggest the possibility of co-development approaches between early stage users and developers through eyetracking.

Verifying the functional behavior of graphical user interface (GUI) applications is essential for reducing post-release issues. In practice, a developer/tester performs this verification by executing a sequence of GUI actions and then witnessing the expected behavior on the GUI screen. An automated witness generator facilitates the verification process. However, creating an unambiguous and monitorable specification for the correct behavior and then generating the correct GUI actions to trigger that behavior is challenging. In this study, we propose FARLEAD2, an automated witness generator that uses unambiguous, monitorable, and easy-to-read staged test scenarios (STSs) to specify expected behavior. FARLEAD2 maximizes its effectiveness and performance using generalized experienced replay (GER) to exploit the experience gathered from previously witnessed scenarios on new, unwitnessed test scenarios. To the best of our knowledge, STS and GER are novel improvements to GUI testing. Our evaluation of Android GUI applications shows that FARLEAD2 effectively generates a witness 95.7 times out of 100 and does it in 520 seconds, on average, indicating that FARLEAD2 is approximately 65 percent faster and 6.3 percent more effective than its best predecessor.

A Computational Model to Predict Aesthetic Quality of Text Elements of GUI

Advanced Robot for Interactive Application (ARIA) is a Humanoid Robotic Head which is capable of mimicking Various Human Facial Expressions. Much work has been done on Implementation of Humanoid Robotic Head with High end systems and Personal Computers (PCs). This paper presents the essential elements necessary for the implementation of Advanced Robot for Interactive Application (ARIA) on UDOO Board. The main aim of the Project was to develop a control system and Graphical User Interface (GUI) for ARIA to deliver real time human facial expressions using embedded board. Humanoid Robotic Head which is capable of mimicking Human Facial Expressions in Real time. Implementation of ARIA involved careful selection of Embedded Board, actuators, control algorithms, motor drivers, operating system, communication protocols, and programming languages. The Board contains a Quad Core A9 Processor and a Controller embedded on it, which are interconnected. In this project the controller is dedicated to control micro servo motors which are controlling eyes, eyebrows and eyelids movements whereas the Processor Handles the Dynamixel motors, GUI and different communication modules.

The paper presents the continuity study from the previous work, which designing the graphical user interface (GUI) for a decision support system (DSS) of driving fatigue. As driving fatigue has been recognised as one of the significant contributory factors to road accidents and fatalities in Malaysia, the author developed the DSS that providing analysis, and proving solutions and recommendations to the road users. In other words, the DSS acts as the advisory and decision-maker tool. In designing the GUI for a DSS, the Django based on Python programming language was used by the authors. There are five main GUI that has been designed in this study: admin GUI user profile and driving information GUI, regression model GUI, risk factor analysis GUI and superuser GUI. Further testing and validation of the graphical user interface for the DSS are suggested before it is used commercially.