GUILGET: GUI Layout GEneration with Transformer

Mobile applications use Accessibility Services to assist users with disabilities (UWDs) in using Graphical User Interface (GUI)-based apps. However, such mobile assistive technologies are not fully secure as these can obtain data from the GUI objects, enhance this data for UWDs and transmit it to a server, thereby potentially exposing sensitive data to the external world. SEAPHISH (SEcuring Accessibility using PHISHing) is a platform aimed towards protecting against such an attack by providing defense by deception. This platform generates a phishing app i.e. an app similar to the original app installed on the user’s smartphone by extracting GUI elements and properties from the original app. A simulation for SEAPHISH can help determine the situations when an attack against a particular app can be performed with a high degree of probability. But performing effective simulations requires a fundamental understanding of the properties of GUI layouts of apps at large. This thesis aims at providing a framework that analyzes GUI layouts and their transitions using a large base of approximately three million Android apps. Various state-of-the-art tools that use different strategies in traversing layouts are explored. We created a framework where the tool Backstage performs static analysis and another tool AndroidRipper performs dynamic analysis of the layouts to help build a GUI model of the app. Using these models, we investigate the layouts of Android apps by collecting statistics on various GUI elements and screens. This investigation enhances SEAPHISH with statistically significant real-world constraints thereby providing defense against malwares and reducing the security vulnerabilities faced by UWDs.

This study investigated the impact of graphical user interface (GUI) design on the efficiency and effectiveness of map-based tasks on mobile devices, using time-based weather data as a case study. Three different GUI designs (button-type, circle-type, and slidebar) were tested in a between-subjects design, with 50 participants completing a set of map-based tasks on each GUI design. The results showed that GUI design significantly affected the effectiveness of map-based tasks. Participants performed better at tasks involving the search for the highest and lowest temperature amplitudes on the button-type GUI whereas the circle-type GUI showed lower effectiveness for tasks involving the search for day temperatures. Analysis of the visual attention distribution based on fixation count revealed that different GUI designs led to different patterns of visual attention. The study highlights the importance of considering GUI design in the development of mobile map applications, particularly for map-based tasks involving time-based data. The study shows that separating the date from the time navigation panel reduces necessary visual focus on the GUI itself and is a valuable insight for future GUI design.

The complexity of software projects and the rapid technological evolution make it such that developers often need additional help and knowledge to tackle their daily tasks. For this purpose, they often refer to online resources, which are easy to access and contain a wealth of information in various formats. Programming screencasts hosted on platforms such as YouTube are one such online resource that has seen a growth in popularity and adoption over the past decade. These screencasts usually have some metadata such as a title, a short description, and a set of tags that should describe what the main concepts captured in the video are. Unfortunately, metadata are often generic and do not contain detailed information about the code showcased in the tutorial, such as the API calls or graphical user interface (GUI) elements employed, which could lead to developers missing useful tutorials. Having a quick overview of the main code elements and GUIs used in a video tutorial can be very helpful for developers looking for code examples involving specific API calls, or looking to design applications with a specific GUI in mind. The aim is to make this information easily available to developers, and propose VID2META, a technique that automatically extracts Java import statements, class names, method information, GUI elements, and GUI screens from videos and makes them available to developers as metadata. VID2META is currently designed to work with Android screencasts. It analyzes video frames using a combination of computer vision, deep learning, optical character recognition, and heuristic-based approaches to identify the needed information in a frame, extract it, and present it to the developer. VID2META has been evaluated in an empirical study on 70 Android programming videos collected from YouTube. The results revealed that VID2META can accurately detect and extract Java and GUI elements from Android programming videos with an average accuracy of 90%.

A Novel Approach to HTML Page Creation Using Neural Network

The contemporary theory of metaphor defines the metaphor as a major and indispensable part of our ordinary, conventional way of conceptualizing the world and it asserts that the metaphors are used to define our relationship with the physical environment and to create a context of communication. The metaphors are the core idioms of the contemporary graphical user interface (GUI) and have an important role in helping users interact with computer systems. The desktop metaphor began with the first GUI Xerox Star and was popularized by the Lisa of Apple Macintosh. Realizing the physical-office metaphor and reflecting the familiar world of the potential users by using icons of file, folder and trashcan, the metaphor-based GUIs made it relatively easy for all to use the computer productively. Nowadays almost all of the software industry is based on the desktop metaphor paradigm. However, besides the researchers that favor the use of metaphors in the development of the system image by accentuating their assistance to learning, there are also other researchers, who reject the metaphor-based design approach in GUIs by referring to their limitations. Considering these discussions and various approaches, this study aims to make an outline of the historical development of metaphor-based GUIs; to determine the current state of metaphor within GUI design by brief examples and finally to identify future prospects and research themes. The study claims that metaphor still remains to be an inevitable part of the GUI to communicate it with the users and despite its potential limitations -rather than rejecting the metaphor totally- it seems better-to try compatible and supplementary solutions like visual formalism.

An important component of Early Warning Systems (EWS) for man-made and natural hazards is the command and control unit’s Graphical User Interface (GUI). All relevant information of an EWS is concentrated in this GUI and offered to human operators. However, when designing the GUI, not only the user experience and the GUI’s screens are relevant, but also the frameworks and technologies that the GUI is built on and the implementation of the GUI itself are of great importance. Implementations differ based on their applications in different domains but the design and approaches to implement the GUIs of different EWS often show analogies. The design and development of such GUIs are performed repeatedly on some parts of the system for each EWS. Thus, the generic GUI framework of a geospatial EWS for tsunamis is introduced to enable possible synergistic effects on the development of other new related technology. The results presented here could be adopted and reused in other EWS for man-made and natural hazards.

This study investigates the key factors influencing the usability of smartphone graphical user interfaces (GUIs) among elderly users in Shanxi Province, China. A total of 1,111 healthy individuals aged 60-75 years participated in the study. Quantitative analysis methods, including descriptive statistics, Pearson correlation, regression analysis, and mediation and moderation analysis, were employed to examine the interactions between user characteristics (age, education, experience, health status) and GUI design factors (interface layout, text and icon design, feedback mechanisms, information presentation) about usability metrics (cognitive load, interaction barriers, user experience). The analysis revealed that both user characteristics and GUI design factors significantly impact usability outcomes. Specifically, GUI design factors were found to mediate the relationship between user characteristics and usability, while user characteristics moderated the effects of GUI design on usability. These findings underscore the necessity of tailored interface design to enhance user experience and promote digital inclusion among elderly populations. Practical recommendations include the use of larger and more readable text, simplified iconography, and clear, immediate feedback to users. Such adjustments can significantly improve usability and foster greater digital inclusion. This research contributes valuable insights to both theoretical understanding and practical applications in the design of age-appropriate mobile interfaces.

This paper introduces an automated grader for Java programs called GUI_Grader that allows students a degree of flexibility in graphical user interface (GUI) design. GUI_Grader allows students to build multi-window Java applications, choose among alternative GUI components, and decide how to order, position and label components. This enables students to practice some aspects of designing their own GUI applications while still providing automated grading `based on a single test plan. The data-driven approach helps to maintain consistency between test plans and program specifications. Testing GUI_Grader on Java assignments from both a first-year and an upper-year course confirms the feasibility of the approach.

PurposeInvestigating one aspect of the potential of firms to market their offerings internationally, this research seeks to explore the impact of that familiar topic – culture – on online international marketing. More precisely, whether the technologies of the web (manifest in the graphical user interface (GUI)) are culturally neutral, allowing for transparent communication between different cultures.Design/methodology/approachUtilising the widely cited cultural dimensions of culture – “uncertainty avoidance” and “power distance” – comparative content analysis for a sample of web sites from two industry sectors (banking and education) across three countries (Germany, Greece, and the United Kingdom) was undertaken.FindingsThe results of the analysis revealed differences in aspects of GUI design between the three country web sites in line with associated cultural dimensions. It also revealed similarities between the three countries in GUI design in apparent contradiction of their associated cultural dimension. This early work suggests, therefore, that, whilst internet‐based technologies exist as a global medium in the sense of connecting individuals, those individuals may still be culturally bound. To truly escape the constraints of time and space, communicating effectively online across boundaries, some acknowledgement that culture does matter is necessary.Research limitations/implicationsFurther research, both in terms of the number of countries and industries, would be needed to confirm the generalisation that culture does indeed matter in terms of web site design (all be it in a qualified way) and, in addition, that the results were not unduly influenced by either the choice of countries or industries. Whilst other researchers should consider applying the methodology to other industries, such as sports, fashion, etc. in order to test further the results of this study, additional research should also look towards different methodologies. For example, analysing web sites utilising Hofstede's other dimensions, or even applying Trompenaars and Hampden‐Turner's dimensions seems desirable.Practical implicationsThis paper shows that culture, at least partially, influences GUI design. The value in the suggested directions made in this paper lies in the possible creation of guidelines for the subsequent development of successful web sites. The work here adds to the limited body of work in this area against which future works may be contrasted.Originality/valueRecognising the apparently obvious virtues of the internet and web in exchanging rich information with distant (international) markets, this work builds on the little work already undertaken in this direction and in doing so contributes to the wider debate in international marketing over standardisation versus adaptation. Specifically, this paper considers the influence of culture in an online context and researches the questions, “Does culture matter online?” and more specifically, “Are the multitude of graphical user‐interfaces of organisational web sites culturally neutral or are the graphical user‐interfaces culturally bound, intervening and influencing the ability to communicate across cultures?” The results are contrasted against predictions for the design of web sites derived from Hofstede's (1980, 2001) seminal works on the dimensions of culture.

The nanoelectronic modeling (NEMO) program is the result of a three-year development effort involving four universities and the former Corporate Research and Development Laboratory of Texas Instruments, now Applied Research Laboratory, Raytheon TI Systems, to create a comprehensive quantum device modeling tool for layered semiconductor structures. Based on the non-equilibrium Green function formalism, it includes the effects of quantum charging, bandstructure and incoherent scattering from alloy disorder, interface roughness, acoustic phonons, and polar optical phonons. NEMO addresses the diverse needs of two different types of users: (i) the engineer/experimentalist who desires a black-box design tool and (ii) the theorist who is interested in a detailed investigation of the physics. A collection of models trade off physical content with speed and memory requirements. Access to this comprehensive theoretical framework is accommodated by a Graphical User Interface (GUI) that facilitates device prototyping and in situ data analysis. We describe a hierarchical software design that allows rapid incorporation of theory enhancements while maintaining a user-friendly GUI, thus satisfying the conflicting criteria of ease of use and ease of development. The theory and GUI modules share data structures that define the device structure, material parameters, and simulation parameters. These data structures may contain general data such as integer and real numbers, option lists, vectors, matrices and the labels for both batch and GUI operation. NEMO generates the corresponding GUI elements at run-time for display and entry of these data structures.

The design of graphical user interfaces (GUI) is a complex activity that binds multiple concerns into meaningful interactive applications. While the definition of multiple orthogonal views greatly facilitates the design of such GUIs, existing GUI frameworks still lack a proper strategy for splitting and binding these views to build complete GUIs. This paper proposes a domain specific language (DSL) called Loa to unify both the definition of orthogonal views and the bindings between these views in the context of GUI development. We present the overall strategy for splitting and binding views based on the use of the Loa DSL and we illustrate the proposed approach on the design of a web-based application.

WebDraw: A machine learning-driven tool for automatic website prototyping

Tools for creating graphical user interfaces (GUIs) hide implementation details associated with changing the position, size or shape of a widget. For example, a GUI designer can change the position of a button without explicitly reimplementing the code that determines if the button has been clicked. We seek to provide a similar experience for moving tangible widgets on physical interactive devices (PIDs). One reason that GUI design tools can hide implementation details is that widgets are modules. In this work in progress, we describe a system for leveraging modularity to hide implementation details associated with changing the position, size and shape of widgets in pid interfaces. We have used the system to design, redesign and fabricate 10 interfaces for 3 example applications. Fundamentaly, working with atoms to make pid interfaces is different than working with pixels to make GUI interfaces--but modularity in pid widgets appears to be a promising way to hide implementation details in pid interface redesign and fabrication.

<div><p class="Els-history-head">The development of rocketry and rocket payload technology is very rapid in the industrial era 4.0. Rockets are usually used as dynamic sensors or carriers for specific missions for sensing and retrieving data from space for meteorological, military, etc. This article aims to design, realize and test a Rocket Payload Monitoring System Based on an Inertial Measurement Unit (IMU) Sensor to monitor the condition of the payload movement behavior. Rocket Payload is a substance carried in a Payload Test Rocket (RUM) in the form of a cylindrical payload. This research focuses on designing a Graphical User Interface (GUI) application to display data in real-time. The type of research used is Research and Development (R&D) with a prototyping development model. The results obtained in the form of Rocket Payload Design and GUI design can be concluded that the GUI application can visualize all sensor and camera data in real-time.</p></div>

During the development of a process or industrial activity, it is essential to have information of the critical variables of the process to know the current state of the system. Monitoring/supervision systems are responsible for collecting such information and presenting it to the user, usually through Graphical User interfaces (GUI) [1]. These interfaces provide the user with visual support to understand the system status and its evaluation. For this reason, interfaces have become a necessary complement in monitoring/supervision systems so, from an academic and practical point of view, it is very beneficial to know how they work and how they are designed.In this context, the widely known MATLAB environment offers the user a toolbox oriented to the design and development of GUI called AppDesigner [2]. This application provides the main components of an industrial monitoring/supervision system, such as input/output fields, graphics or buttons [3]. This tool is divided into two tabs, one for the design of the graphical aspect and the other for the programming of the components that make up the GUI. The function performed by the components are programmed by means of scrips in MATLAB language. This feature gives the GUI great versatility, limiting the functions it can perform to the limitations of the programming language.As part of MATLAB, AppDesigner can interact with other MATLAB environments and toolboxes, such as Simulink or OPC toolbox. It must be noted that handling communication protocols is a requirement for data sharing with industrial automation devices like programmable controllers [4]. It is also possible to communicate with external programs such as Excel to perform different functions e.g., data exchange.Due to the characteristics indicated above and the wide use of MATLAB in the academic field, in this contribution the utilization of this application as an educational tool for teaching monitoring/supervision systems in industrial environments is studied.