Evolving User Interfaces From Within Self-supporting Programming Environments

Before a new structure can be built, it must be designed. This design phase is a very important step in the building process. The total cost of the structure and its structural performance are largely dependent on the structural design process. The impact of decisions on the design process is initially high and declines as the design matures. However, few computational tools are available for the conceptual design phase; thus, an opportunity exists to create such tools. In the conventional workflow, the architect uses geometric modeling tools and the engineer uses structural analysis tools in sequential steps. Parametric modeling tools represent an improvement to this workflow, as structural analysis plug-ins are available. This allows the architect or engineer to receive structural feedback at an earlier stage, but still as a sequential step to the geometric modeling. The present work aims to improve this workflow by integrating structural feedback with geometric modeling.The user interfaces of conceptual design tools should be interactive and agile enough to follow the designer’s iterative workflow. Direct manipulation involves human-computer interaction, which enables an interactive user interface. In this user interface style, users can directly manipulate on-screen objects using real-world metaphors, which engages the users with their task and encourages further explorations. This is achieved by reducing the perceptual and cognitive resources required to understand and use the interface. New technologies have opened up the possibility of creating new design tools that make use of very direct manipulation. This possibility is further explored in this thesis through the development of two such applications. The first application makes use of multi-touch tablets. The multi-touch interface has literally closed the gap between humans and computers, enabling very direct manipulation interactions with two-dimensional user interfaces. The developed application is an interactive conceptual design tool with real-time structural feedback that allows the user to quickly input and modify structural models through the use of gestures. The second application extends these concepts and ideas into a three-dimensional user interface using an input device named the Leap Motion Controller.

Preferred mental models for direct manipulation and command-based interfaces

Novices face many barriers when learning to program a computer, including the need to learn both a new syntax and a model of computation. By constraining syntax and providing concrete visual representations on which to operate, direct manipulation programming environments can potentially lower these barriers. However, what if the ultimate learning goal of the novice is to be able to program in conventional textual languages, as is the case for introductory computer science students? Can direct manipulation programming environments lower the initial barriers to programming, and, at the same time, facilitate positive transfer to textual programming? To address this question, we designed a new direct manipulation programming interface for novices, and conducted an experimental study to compare the programming processes and outcomes promoted by the direct manipulation interface against those promoted by a textual programming interface. We found that the direct manipulation interface promoted significantly better initial programming outcomes, positive transfer to the textual interface, and significant differences in programming processes. Our results show that direct manipulation interfaces can provide novices with a “way in” to traditional textual programming.

This paper reports a human factors study comparing the usability of command language and direct manipulation interfaces for computer numerically-controlled (CNC) milling machines. The operator-machine interfaces were assessed in terms of task completion time, operator retention of training knowledge, and subjective preference judgements. Four factors were manipulated in the study: (1) work session (two sessions separated by a one week interval); (2) level of operator expertise (novices and experts); (3) programming task (copy and modify); and (4) trial number. The one week interval between work sessions degraded task completion time for the command language interface, but not for the direct manipulation interface. Moreover, operators knowledge of CNC programming was retained better with the direct manipulation interface as compared to the command language interface. Additionally operators' subjective ratings indicated a strong preference for the direct manipulation interface rather than the command language interface across several categories of usability attributes.

This paper describes a system for constructing graphical user interfaces following a two-view paradigm: one view contains a textual representation of the interface in a special-purpose, little language, and the other view contains a direct manipulation, interactive editor for the user interface. The user interface can be edited in either view, and the changes are reflected in the other view. The language allows dialog boxes to be expressed in a simple and natural way, and has a well-defined mapping into the interactive editor. A base set of interactors is currently available, but the system can be easily extended with more interactors. We believe this approach to building user interfaces combines the advantages of the direct manipulation, WYSIWYG approach with the advantages of the textual, descriptive approach, and does not suffer from the limitations of either approach.

Mixing consoles are the primary tool used in the mixing of multi‐track recordings of musical instruments and feature an equalisation (EQ) section to manipulate the spectral characteristics of each channel. Despite numerous technological advances, modern embodiments of user interfaces for mixing continue to predominantly follow the traditional linear channel strip paradigm. The aim of this research is to question and reconsider one aspect of the mixing desk user interface, namely, the EQ user interface. The research methodology involved evaluating example mixing consoles, their EQ sections and considering a range novel mixing and EQ interfaces. A literature review of Human Computer Interface (HCI) fundamentals established current design and evaluation approaches. This is supplemented by case studies of the design and evaluation of several user interfaces for music technology applications. In line with HCI theory an analysis of the EQ task was conducted and an expert user consulted throughout the design process, simulating a range of scenarios to refine and develop paper prototypes into workable designs. A corrective EQ was developed that was favoured in evaluation over a typical existing Digital Audio Workstation (DAW) EQ. The novel features of the corrective EQ user interface were spectral data visualised as a static spectral plot with five peak frequencies presented to the user allowing the user to directly manipulate the spectral plot which simultaneously provided real‐time visual feedback. A multi‐track EQ was developed that was favoured in evaluation over an existing DAW EQ for multi‐track EQ tasks. The multi‐track EQ user interface featured two tracks displayed simultaneously and added novel direct manipulation method for performing mirrored EQ in the form of ghost nodes representing peaks from the other track on a track’s spectral plot. A similar visual and direct manipulation method for performing band pass and low and high shelf filtering was also included.

Java Beans provides a robust mechanism for combining software components. However, to a large extent Java Beans components are currently not being combined to create documents in an environment such as a word processor or illustration program. This is not due to a weakness in the Beans property model, but rather to limitations in how Java Beans' user interfaces are currently implemented. Verdantium is a prototype Java-based API (for JDK 1.2.2 and above) for document embedding that takes a novel approach to implementing software components. This environment retains the property-based semantics from Java Beans, but implements different approaches for on-screen presentation, visual embedding, document printing, and persistence. Fully functional applications (i.e., word processors, illustration programs, equation editors, etc.) can be implemented as individual software components that can either run independently or be embedded in other applications using the same API.

Distributed multimedia applications supported by a global electronic infrastructure have tremendous potential for providing users with customized communication and computation environments. Since communication and computation requirements vary by context and change dynamically, it is unlikely that offthe-shelf applications will anticipate the needs of all users. Therefore, empowering end-users to create their own customized applications for both communication and computation is an important challenge. This dissertation presents several mechanisms that enable end-users to create and configure distributed multimedia applications, including end-users construction direct manipulation graphical users interface (GUIs) and application management of distributed multimedia applications over the Internet.

The language reading direction is probably one of the most determinant factors influencing the successful internationalization of graphical user interfaces, beyond their mere translation. Western languages are read from left to right and top to bottom, while Arabic languages and Hebrew are read from right to left and top to bottom, and Oriental languages are read from top to bottom. In order to address this challenge, we introduce flippable user interfaces that enable the end user to change the reading direction of a graphical user interface by flipping it into the desired reading direction by direct manipulation. This operation automatically and dynamically changes the user interface layout based on a generalized concept of reading direction and translates it according to the end user's preferences.

A reflective characterisation of occasional user

We characterize and demonstrate how the principles of direct manipulation can improve interaction with large language models. This includes: continuous representation of generated objects of interest; reuse of prompt syntax in a toolbar of commands; manipulable outputs to compose or control the effect of prompts; and undo mechanisms. This idea is exemplified in DirectGPT, a user interface layer on top of ChatGPT that works by transforming direct manipulation actions to engineered prompts. A study shows participants were 50% faster and relied on 50% fewer and 72% shorter prompts to edit text, code, and vector images compared to baseline ChatGPT. Our work contributes a validated approach to integrate LLMs into traditional software using direct manipulation. Data, code, and demo available at https://osf.io/3wt6s.

Dance appreciation is a creative and complex spiritual activity process of the unity of feeling and understanding, affection and understanding. Development of information technology and mobile terminals provide a new way to dance appreciation. In order to develop dance appreciation video system, this paper designed it based on the MVC mode and provided technical support for software development. Firstly, studied the MVC mode, using graph represented the relationship among modes, views and controllers; and then designed layered system architecture consisting of user layer, service layer and management layer; finally, with the live video of one of the core program designed the program flow and illustrated the implementation process. The results shown that forcibly defining controller, model, view and other modules of application improved the readability of application and the reusability of code.

The production of graphical interfaces is often hindered by the difficulty designers find in visualising the interface. This paper considers a user interface tool which allows the construction of graphical interfaces by direct manipulation. It also considers the specification of interfaces using an object oriented approach. Finally it considers how the results of evaluation of the user interface can be linked to the design specification for the improvement of subsequent interface prototypes. The tool is part of the Alvey ECLIPSE integrated project support environment.

Object-oriented architectures for Graphical User Interfaces (GUI’s) model the interface as a composition of interacting Objects which each present an internal state to the user and provide operations on that state. Two such architectures are Model-View-Controller (MVC) and Presentation-Abstraction-Control (PAC).We use Object-Z to describe examples of the MVC and PAC architectures. We compare the characteristics of the architectures and consider the efficacy of Object-Z as a tool for modelling user-interface architectures.

Construction materials play a crucial role in the project delivery process. However, the process of requisition for the materials on construction site if not handled appropriately can adversely affect the performance of quality, cost, and time. Therefore, the study was aimed at developing an automated materials requisition system which will make the supply chain on construction sites seamless and more effective. A use case diagram and an activity block diagram helped to understand the users and functionalities of the material requisition platform. In addition, using different user interface and a database system including a programming language to connect them, the study developed an automated Web-based material requisition system for construction firms using the model view controller (MVC) model. The MVC model comprised of using MySQL, HTML, and PHP, respectively, in the design. The automated materials requisition system was tested by sending material requisition through the supply chain of a construction firm. Result of the automated system was presented via screenshots of the Web-based platform. In conclusion, any construction firm can register on the platform and make use of the automated materials requisition system in order to maximize the productivity and optimize the use of ICT in their materials’ supply chain process.