Improving object-oriented user interfaces with constraints

User interfaces represent a major part in software products. Generally, it is difficult to document the content and the structure of the user interface. However, there is a need to save and preserve the overall state and structure of the user interface for later communication and modifications in requirements, design and implementation. The Graphical User Interface (GUI) storage format should be universal and can be transferred from one application to another (such as XML). GUI states are usually defined as the overall combination of properties of all the components or widgets of the GUI This paper suggests an alternative back end representation of user interfaces and their state from their original format within the applications. The user interface model is transformed to a schema saved in an XML file. The schema structure represents the user interface state. This state considers only the structure of the user interface and ignores controls' specific properties. XML format is widely used and accepted by many applications. It is the infrastructure language for web pages and database management systems. User interface documentation is useful for future evaluation and comparison, useful for stakeholders' communication. It is also for some GUI testing activities such as regression testing where we trigger testing if there is a change in the user interface.

AI (Artificial Intelligence) and GUI (Graphical User Interface) are the two most cutting-edge technologies. These technologies individually can make a comprehensive change in industries and alter economic and social interactions and connections. In general, Now-a-days AI is integrated with human technology and used in some wide range of sectors like healthcare, finance to education, banking, manufacturing, transportation, etc. GUI is used in gaming, designing, AR (Augmented reality) and VR (Virtual reality), media, smart home devices, etc., Integration of Artificial Intelligence and Graphical User Interface is beneficial for both users and app developers. The convergence of AI and GUI gives dynamic applications like Data Visualization, Insights, Smart assistants, Adaptions, Navigations, and Predictive analytics. This integration will increase the ease of access for new information to users. Now we are facing some general problems based on these technologies like Adaptive User Interfaces, Automated user interface testing, AI-Enhanced data visualization, Intelligent GUI Design Assistance, Real-Time Emotion Recognition for GUI Adaptation, etc., User interfaces are often designed with a one-size-fits-all approach, which does not account for individual user preferences and abilities. Designing perceptive and attractive GUIs requires significant space and time. This chapter focus on how can we develop a UI (User Interface) with base of AI which can be capable of solving the above obstacles and gives the simple objectives and technical solutions. By this integration we can execute some future projects like AI-Enhanced Customer Support Portals, Adaptive Gaming Interfaces, Interactive Financial Management Systems, and more.

Implementation of a Chatbot System using AI and NLP

This research project investigates the connections between user identity, product emotion and interface design. It investigates whether mobile communication devices can better reflect user identity and lifestyle through the provision of a customizable Graphical user interface (GUI) that users can actively manipulate. The study proposes a user and design-driven interaction process whereby users can easily personalize the interface of their devices to reflect their idiosyncratic needs and preferences, the designed interface making interaction more individually useful, meaningful and empowering. The project extends the scope of end user customization beyond basic cosmetic changes to the physical appearance of the GUI to the modification of the actual process of interaction. The research focuses on the capacity to employ the user interface to support more intimate and flexible patterns of use that foster personal identification and emotional investment with devices, as against those that are preordained, device-driven and technologically deterministic.

Mobility for graphical user interfaces (GUIs) is a challenging problem, as different GUIs need to be constructed for different device capabilities and changing context, preferences and users’ locations. GUI developers frequently create multiple user interface versions for different devices. The solution lies in using a single, abstract, user interface description that is used later to automatically generate user interfaces for different devices. Various techniques are proposed to adapt GUIs from an abstract specification to a concrete interface. Design-time techniques have the possibility of creating better performing GUIs but, in contrast to run-time techniques, lack flexibility and mobility. Run-time techniques’ mobility and autonomy can be significantly improved by using mobile agent technology and an indirect GUI generation paradigm. Using indirect generation enables analysis of computer-human interaction and application of artificial intelligence techniques to be made at run-time, increasing GUIs’ performance and usability.

Integrating the user interface in an object-oriented measurement system

This paper presents a distributed, multi-user virtual reality testbed for a mobile robot. The main components of the testbed include user interface, networking, simulation and robot control. The user interface is implemented as a combination of a virtual environment and a standard graphical user interface (GUI). The virtual environment is developed using the Virtual Reality Modeling Language (VRML) and it corresponds to the laboratory environment where the robot is placed and tested. The standard GUI is implemented using Java Programming language. The World Wide Web (WWW) browser is used to integrate the virtual environment (embedded VRML) and the standard GUI (Java applet) in a single user interface. Users can access a dedicated WWW server and download the user interface. Networking component (communication among users and robots) is implemented using the Distributed Interactive Simulation (DIS) protocol. The existing robot development system has been utilized to provide simulation capabilities. A separate stand-alone Java application has been developed to communicate and control the robot using a wireless Ethernet link. In addition to user's direct control, control algorithms (e.g. fuzzy logic based) can be used to replace the user. A Nomad200 mobile robot has been used for initial measurements and evaluations of the testbed.

XML Based Graphical User Interface Editor and Runtime Parser for ISO 11783 Machine Automation Systems

Analyses some of the components of graphical user interfaces (GUI), applied to CD-ROM databases (windows, dialogue boxes, menus, commands, buttons, check boxes and icons) and remarks on the degree to which these are based on Windows software. Notes the constraints to GUIs that still remain even after databases have appeared in the marketplace: particularly mouse based prob lems. Discusses ways of evaluating GUIs and their specific components and reports a study of databases in which these types of interfaces are used. Notes the significant variability between products in the way that they apply GUIs and discusses whether database users would benefit from a certain degree of standardization in the basic interface structure using GUIs, or whether users would prefer greater control over the design of the local user interface. Concludes that the introduction of Windows based CD-ROM databases with graphical user interfaces will increasingly revolutionize the design of such data bases. Makes 3 recommendations that would aim to enhance the user friend liness of CD-ROM interfaces: research to develop criteria for evaluating interfaces in the GUI environment; standardization of interface design between some products, particularly for novice users; and facilities to allow experienced users to develop and tailor their own interface design.

The World Wide Web is increasingly becoming an important medium for business growth. The migration of existing applications to the web is a core problem in software reengineering, since applications are frequently deemed to be useful when their functionalities are accessible via the Web. A substantial class of applications that could benefit from migration to the Web is that of Java Graphical User Interface (GUI) desktop applications. An emerging approach to Web migration is interaction-based reengineering, in which the User Interface (UI) of the existing application is wrapped with a software layer that exports a Web accessible interface and hides the original one. A major barrier to such migration approaches is the reverse engineering of the User Interface, which is usually a time consuming and highly manual task. We present a new approach to automatically Reverse Engineer various structural and behavioral features of a GUI from an executing Java-Swing desktop application. The approach is based on source code independent techniques, such as Aspect J. In this paper, we describe our GUI reverse engineering approach and we illustrate it with a case study.

The crew-centered cockpit design (CCCD) program is developing a formal, structured process to improve the design, analysis, and testing of cockpits. Veda Incorporated is working with the CCCD Program Office to provide an enhanced and validated crew-centered system design process (CSDP) and a cockpit design system (CDS) toolset to meet this need. A key component of the toolset is a reconfigurable engineering design simulator that can be used to model cockpit display concepts quickly and easily. This simulator uses an avionics instrument development system to prototype displays that require high-resolution, high-performance, real-time graphics. This paper provides details of the development of a cockpit display application from design through integration. Using an object-oriented approach and graphical user interface, a crew system designer can rapidly model display concepts. The resulting display software can be distributed to other users, and can then be executed (with or without modifications) on a Silicon Graphics system without the need for a run-time license. This paper also describes a distributed, scalable hardware and software architecture that is being used to support the simulator. The architecture features the Unix operating system and a homogeneous computing environment of Silicon Graphics workstations that rely heavily on replicated shared memory (also known as reflective memory) and Ethernet networks.

Mobile applications use Accessibility Services to assist users with disabilities (UWDs) in using Graphical User Interface (GUI)-based apps. However, such mobile assistive technologies are not fully secure as these can obtain data from the GUI objects, enhance this data for UWDs and transmit it to a server, thereby potentially exposing sensitive data to the external world. SEAPHISH (SEcuring Accessibility using PHISHing) is a platform aimed towards protecting against such an attack by providing defense by deception. This platform generates a phishing app i.e. an app similar to the original app installed on the user’s smartphone by extracting GUI elements and properties from the original app. A simulation for SEAPHISH can help determine the situations when an attack against a particular app can be performed with a high degree of probability. But performing effective simulations requires a fundamental understanding of the properties of GUI layouts of apps at large. This thesis aims at providing a framework that analyzes GUI layouts and their transitions using a large base of approximately three million Android apps. Various state-of-the-art tools that use different strategies in traversing layouts are explored. We created a framework where the tool Backstage performs static analysis and another tool AndroidRipper performs dynamic analysis of the layouts to help build a GUI model of the app. Using these models, we investigate the layouts of Android apps by collecting statistics on various GUI elements and screens. This investigation enhances SEAPHISH with statistically significant real-world constraints thereby providing defense against malwares and reducing the security vulnerabilities faced by UWDs.

Nowadays, touchable displays are widely used in most embedded systems, including mobile phones and other hand-held devices. Although interactive user interfaces are intuitively required for these systems, most of interactive graphical user interface implementations usually need lots of system resources. For low-tier embedded systems, we have presented that the text-based user interfaces are more suitable, as shown in our previous works. In this paper, we present an object-oriented approach to provide a full-scale text-based user interface scheme, especially on the low-tier embedded systems. Our application user interface implementation is very simple and intuitive, with few amount of computing resource. It naturally supports multiple overlapping windows, pop-up menus, pull-down menus, dialog boxes, and other user interface components, with graceful object-oriented application programming interfaces. We also provide large size text output for emphasis purposes. All the systems are now available as a software library, and can be used for wide range of applications.

apART reflects the structure of an open, distributed environment. According to the general trend in the area of imaging, network-capable, general purpose workstations with capabilities of open system image communication and image input are used. Several heterogeneous components like CCD cameras, slide scanners, and image archives can be accessed. The system is driven by an object-oriented user interface where devices (image sources and destinations), operators (derived from a commercial image processing library), and images (of different data types) are managed and presented uniformly to the user. Browsing mechanisms are used to traverse devices, operators, and images. An audit trail mechanism is offered to record interactive operations on low-resolution image derivatives. These operations are processed off-line on the original image. Thus, the processing of extremely high-resolution raster images is possible, and the performance of resolution dependent operations is enhanced significantly during interaction. An object-oriented database system (APRIL), which can be browsed, is integrated into the system. Attribute retrieval is supported by the user interface. Other essential features of the system include: implementation on top of the X Window System (X11R4) and the OSF/Motif widget set; a SUN4 general purpose workstation, inclusive ethernet, magneto optical disc, etc., as the hardware platform for the user interface; complete graphical-interactive parametrization of all operators; support of different image interchange formats (GIF, TIFF, IIF, etc.); consideration of current IPI standard activities within ISO/IEC for further refinement and extensions.

An alternative solution for Indosport, considering the increasingly advanced smartphone development and also followed by the rapid growth of its users in Indonesia. The purpose of this study project is to produce a product in the form of a user interface (UI) and user experience (UX) design in helping to realize targeted mobile applications and not forgetting to apply the principles of visual communication design namely unity and variety, hierarchy, contrast, proportion and balance. The UI and UX design methods go through several stages of work in sequence: pre-production process, production process, and post-production. This design results in UI and UX designs that are equipped with user flow, interface designs, illustrations and prototypes. The main page of the Indosport mobile application is News, Activity, Calendar, and Profile which later on these pages there are several features that are different from the previous page. The making of this UI uses several visual languages such as medium close up, full shot, long shot, close up and medium long shot in the image section contained in the article as well as the depiction of field styles used to almost all visual UI elements. Using several principles in design, namely time to learn, speed of performance, rate of errors by users, retention over time, and subjective satisfaction. The resulting design forms will be distributed directly to the Research and Development division and the IT division from Indosport to be made a native.