Designing a compelling user interface for morphing

Controlling assistive robots can be challenging for some users, especially those lacking relevant experience. Augmented Reality (AR) User Interfaces (UIs) have the potential to facilitate this task. Although extensive research regarding legged manipulators exists, comparatively little is on their UIs. Most existing UIs leverage traditional control interfaces such as joysticks, Hand-Held (HH) controllers and 2D UIs. These interfaces not only risk being unintuitive, thus discouraging interaction with the robot partner, but also draw the operator’s focus away from the task and towards the UI. This shift in attention raises additional safety concerns, particularly in potentially hazardous environments where legged manipulators are frequently deployed. Moreover, traditional interfaces limit the operators’ availability to use their hands for other tasks. Towards overcoming these limitations, in this article, we provide a user study comparing an AR Head-Mounted Display (HMD) UI we developed for controlling a legged manipulator against off-the-shelf control methods for such robots. This user study involved 27 participants and 135 trials, from which we gathered over 405 completed questionnaires. These trials involved multiple navigation and manipulation tasks with varying difficulty levels using a Boston Dynamics’s Spot, a 7 df Kinova robot arm and a Robotiq 2F-85 gripper that we integrated into a legged manipulator. We made the comparison between UIs across multiple dimensions relevant to a successful human–robot interaction. These dimensions include cognitive workload, technology acceptance, fluency, system usability, immersion and trust. Our study employed a factorial experimental design with participants undergoing five different conditions, generating longitudinal data. Due to potential unknown distributions and outliers in such data, using parametric methods for its analysis is questionable, and while non-parametric alternatives exist, they may lead to reduced statistical power. Therefore, to analyse the data that resulted from our experiment, we chose Bayesian data analysis as an effective alternative to address these limitations. Our results show that AR UIs can outpace HH-based control methods and reduce the cognitive requirements when designers include hands-free interactions and cognitive offloading principles into the UI. Furthermore, the use of the AR UI together with our cognitive offloading feature resulted in higher usability scores and significantly higher fluency and Technology Acceptance Model scores. Regarding immersion, our results revealed that the response values for the AR Immersion questionnaire associated with the AR UI are significantly higher than those associated with the HH UI, regardless of the main interaction method with the former, i.e., hand gestures or cognitive offloading. Derived from the participants’ qualitative answers, we believe this is due to a combination of factors, of which the most important is the free use of the hands when using the HMD, as well as the ability to see the real environment without the need to divert their attention to the UI. Regarding trust, our findings did not display discernible differences in reported trust scores across UI options. However, during the manipulation phase of our user study, where participants were given the choice to select their preferred UI, they consistently reported higher levels of trust compared to the navigation category. Moreover, there was a drastic change in the percentage of participants that selected the AR UI for completing this manipulation stage after incorporating the cognitive offloading feature. Thus, trust seems to have mediated the use and non-use of the UIs in a dimension different from the ones considered in our study, i.e., delegation and reliance. Therefore, our AR HMD UI for the control of legged manipulators was found to improve human–robot interaction across several relevant dimensions, underscoring the critical role of UI design in the effective and trustworthy utilisation of robotic systems.

This chapter will show you how to use and specialise UML diagrams for describing the user interfaces of a software system. In order to accomplish the description of user interfaces, the proposed technique considers three specialised UML diagrams called user-interaction, user-interface, and GUI-class diagrams, which will be built following a model-driven development (MDD) perspective. These diagrams can be seen as the UML-based UI models of the system. In addition, this chapter is concerned with code-generation to implement the user interfaces of the system by using GUI-class diagrams and user-interaction diagrams. A case study of an Internet book shopping system is introduced in this chapter to proof and illustrate the proposed user interaction and interface design technique.

An alternative solution for Indosport, considering the increasingly advanced smartphone development and also followed by the rapid growth of its users in Indonesia. The purpose of this study project is to produce a product in the form of a user interface (UI) and user experience (UX) design in helping to realize targeted mobile applications and not forgetting to apply the principles of visual communication design namely unity and variety, hierarchy, contrast, proportion and balance. The UI and UX design methods go through several stages of work in sequence: pre-production process, production process, and post-production. This design results in UI and UX designs that are equipped with user flow, interface designs, illustrations and prototypes. The main page of the Indosport mobile application is News, Activity, Calendar, and Profile which later on these pages there are several features that are different from the previous page. The making of this UI uses several visual languages such as medium close up, full shot, long shot, close up and medium long shot in the image section contained in the article as well as the depiction of field styles used to almost all visual UI elements. Using several principles in design, namely time to learn, speed of performance, rate of errors by users, retention over time, and subjective satisfaction. The resulting design forms will be distributed directly to the Research and Development division and the IT division from Indosport to be made a native.

Nowadays most of people of developed economy countries interact with software every day. As a result of computer systems expansion to all scope of people's activity the problem of transition from visual and command interfaces to natural language user interfaces is thrown into the sharp relief. Computational linguistics and natural language processing methods are described in this article. Represent a formal definition of a dialogue system as a Markov decision process. Methods of natural language manipulation are applied in machine translation software systems, systems of search and exchange the data, text annotation and expert systems. Prototype of natural language user interface to structured data source is developed. As a result it is convert natural language user's query to SQL query to database. Also showed application of Levenshtein distance for processing natural language query to SQL query to database. Natural language user interface is created to predefined subject field. User interface interacts to database that contain information about existent program libraries and frameworks. Consequently, using natural language processing methods it is possible to develop user natural language user interface providing capability to interact with machine.

At the time of this paper, the advances in light field technology already offer 3D displays that immerse the users without the need for additional viewing devices. Despite the numerous advantages and attractive capabilities of such glasses-free 3D displays, their user interface methods are quite complicated and they are currently underwhelming when compared to conventional 2D displays, due to the fact that visual feedback can only be rendered sharply on the emission surface of light field displays. The sharp rendering of user interfaces is a necessity, as blur may hinder their fundamental functions. When it comes to 2D displays, many user interaction techniques and interfaces have been devised. Rendering a user interface on a 2D display could be done in various ways, such as rendering overlays on top of the rendered scene, or by using billboards. These are extensively used in modern video games. User interaction methods have proven their importance and added efficiency to virtual environments throughout the years. Due to their overall value and usefulness, interaction techniques develop immediately as new types of displays arise. With the recent advancements in visualization technologies, user interfaces have been redesigned for use in AR, VR and MR visualization. This includes on-screen augmentation, which enables interaction with visual content on the screen. Although light field displays contain immense potentials, only basic user interfaces have been devised thus far, including FOX (Focus Sliding surface), which grants users the option to scale and to rotate 3D objects. In this paper, we visualize the theater model on real light field displays and we test the different interactions by means of a monitor room. The theatre model is analogous to real-life theatres, where viewers may observe the theatrical presentation on the stage from various angles. The motivation to choose the theater model was the fact that light field visualization similarly allows multiple simultaneous viewers within its field of view, in which the content can be observed in an angle- dependent manner. Moreover, from the users’ perspective, the theater model is thus familiar and it provides high-quality visual feedback. Furthermore, theater stages encompass a lot of interactions, including rigging and flying systems, pulleys, rotating stages, lights, curtains etc. In order to test the different interaction methods on light field displays, a theater model depicting the virtual environment was implemented. Methods for rendering the monitor room and the results of the interactions are discussed in the paper, illustrated by images of the actual visualization on light field displays. It is shown that producing plausible results with no noticeable visual artifacts is challenging, yet possible. The scientific contributions of the paper also highlight the various novel user interfaces for future light field systems and services.

Quickly accessing the contents of a video is challenging for users, particularly for unstructured video, which contains no intentional shot boundaries, no chapters, and no apparent edited format. We approach this problem in the domain of lecture videos though the use of machine learning, to gather semantic information about the videos; and through user interface design, to enable users to fully utilize this new information. First, we use machine learning techniques to gather the semantic information. We develop a system for rapid automatic semantic tagging using a heuristic-based feature selection algorithm called Sort-Merge, by using large initial heterogeneous low-level feature sets (cardinality greater than 1K). We explore applying Sort-Merge to heterogeneous feature sets though two methods: early fusion and late fusion. Each takes different approaches to handling the different kinds of features in the heterogeneous set. We determine the most predictive feature sets for key-frame filters such as “has text”, “has computer source code”, or “has instructor motion”. Specifically we explore the usefulness of Harr Wavelets, Fast Fourier Transforms, Color Coherence Vectors, Line Detectors, Ink Features and Pan/Tilt/Zoom detectors. For evaluation, we introduce a “keeper” heuristic for feature sets, which provides a method of performance comparison against a baseline. Second, we create a user interface to allow the user to make use of the semantic tags we gathered though our computer vision and machine learning process. The interface is integrated into an existing video browser, which detected shot-like boundaries and presented a multitimeline view. The content within shot-like boundaries is represented by frames to which our new interface applies the generated semantic tags. Specifically, we make accessible the semantic concepts of 'text', 'code', 'presenter', and 'person motion.' The tags are detected in the simulated shots using the filters generated with our machine learning approach and are displayed to users using a user-customizable multi-timeline view. We also generate tags based on ASRgenerated transcripts that have been limited to the words provided in the index of the course text book. Each of these occurrences is aligned with the simulated shots. Each spoken word becomes a tag analogous to the visual concepts. A full Boolean algebra over the tags is provided to enable new composite tags such as 'text or code, but no presenter.' Finally, we quantify the effectiveness of our features and our browser through user studies, both observational and task driven. We find that users that use the full suite of tools performed a search task in 60% of the time of users without access to tags. We find that when users are asked to perform search tasks they follow a nearly fixed pattern of accesses, alternating between the use of tags and Keyframes, or between the use of Word Bubbles and the media player. Based on user behavior and feedback, we redesigned the interface to group spatially interface components that are used together, removed un-used components, and redesigned the display of Word Bubbles to match that of the Visual Tags. We found that users strongly preferred the Keyframe tool, as well as both kinds of tags. Users also either found the algebra very useful or not useful at all.

In recent years, natural language machine interfaces have become increasingly common. These interfaces allow for more intuitive communication with machines, reducing the complexity of interacting with these systems and enabling their use by non-expert users. Most of these natural language interfaces rely on speech, including such well-known devices as the iPhone's Siri application, Cortana, Amazon's Alexa and Echo devices, and others. Given their intuitive functionality, natural language interfaces have also been investigated as a method for controlling unmanned aerial vehicles (UAVs), allowing non-subject matter experts to use these tools in their scientific pursuits. This paper examines a speech-based natural language interface for defining UAV trajectories. To determine the efficacy of this interface, a user study is also presented that examines how users perform with this interface compared to a traditional mouse-based interface. The results of the user study are described in order to show how accurately users were able to define trajectories as well as user preference for using the speech-based system both before and after participating in the user study. Additional data are presented on whether users had previous experience with speech-based interfaces and how long they spent training with the interface before participating in the study. The user study demonstrates the potential of speech- based interfaces for UAV trajectory generation and suggests methods for future improvement and incorporation of natural language interfaces for UAV pilots.

Mixed prototyping is an industrial practice that combines virtual and real components in order to realize a prototype of a product used to evaluate and assess the design choices. Recently, mixed prototypes have been also used to assess the usability of products interface. This particular application arises several problems related to the devices and the interaction techniques that, better than others, allow a natural interaction with the mixed prototype. This paper presents a mixed reality environment for usability evaluation that deals with two specific problems of this kind of application: the occlusion between real and virtual objects and the interpretation of the user’s gestures while he/she is interacting with the elements of the product interface. In particular we propose a technique able to manage both the problems by using only commodity hardware and video processing algorithms, thus avoiding the use of expensive data-gloves and tracking devices. The proposed approach has been validated through a user study addressed to establish whether and to what extent the augmented reality devices and the techniques proposed may distort the usability assessment of the product. Moreover, the user study compares the mixed reality environment adopted in this study with a classical virtual reality set-up.

While natural language interfaces (NLIs) are increasingly utilized to simplify the interaction with data visualization tools, improving and adapting the NLIs to the individual needs of users still requires the support of developers. ONYX introduces an interactive task learning (ITL) based approach which enables NLIs to learn from users through natural interactions. Users can personalize the NLI with new commands using direct manipulation, known commands, or by combining both. To further support users during the training process, we derived two design goals for the user interface, namely providing suggestions based on sub-parts of the command and addressing ambiguities through follow-up questions and instantiated them in ONYX. In order to trigger reflections and gain feedback on possible design trade-offs of ONYX and the instantiated design goals, we performed a formative user study to understand how to successfully integrate the suggestions and follow-up question into the interaction.

It takes movie camera operators years of professional training to follow an object in an aesthetically pleasing way, both by classical means (boom, slider) and by drones. As this complex task requires a high workload and situation awareness while controlling the camera, an uncluttered and efficient user interface (UI) is preferred. The emergence of mobile devices and motion control devices incorporating automation made touch-based UIs attractive to operators. Much work has already been done on UI adaptation strategies. However, little work is trying to solve the problem of combining manual control and automation within a UI. Especially with a central premise of minimising occlusion and visual clutter in a cinematic context. We, therefore, conducted a first user study (N=15) to evaluate different design alternatives regarding occlusion and preference. Afterwards, we created a functional prototype of the most promising design. To further reduce the occlusion we applied a progressive reduction adaption strategy. We evaluated the influence of different reduction levels on workload, control, creativity support and precision in a second user study (N=24). While we could reduce the clutter, due to our design decisions we found no negative effects affecting the measured variables.

BackgroundIndividuals living with sickle cell disease (SCD) may benefit from a variety of disease-modifying therapies, including hydroxyurea, voxelotor, crizanlizumab, L-glutamine, and chronic blood transfusions. However, allogeneic hematopoietic stem cell transplantation (HCT) remains the only nonexperimental treatment with curative intent. As HCT outcomes can be influenced by the complex interaction of several risk factors, HCT can be a difficult decision for health care providers to make for their patients with SCD.ObjectiveThe aim of this study is to determine the acceptability and usability of a prototype decision support tool for health care providers in decision-making about HCT for SCD, together with patients and their families.MethodsOn the basis of published transplant registry data, we developed the Sickle Options Decision Support Tool for Children, which provides health care providers with personalized transplant survival and risk estimates for their patients to help them make informed decisions regarding their patients’ management of SCD. To evaluate the tool for its acceptability and usability, we conducted beta tests of the tool and surveys with physicians using the Ottawa Decision Support Framework and mobile health app usability questionnaire, respectively.ResultsAccording to the mobile health app usability questionnaire survey findings, the overall usability of the tool was high (mean 6.15, SD 0.79; range 4.2-7). According to the Ottawa Decision Support Framework survey findings, acceptability of the presentation of information on the decision support tool was also high (mean 2.94, SD 0.63; range 2-4), but the acceptability regarding the amount of information was mixed (mean 2.59, SD 0.5; range 2-3). Most participants expressed that they would use the tool in their own patient consults (13/15, 87%) and suggested that the tool would ease the decision-making process regarding HCT (8/9, 89%). The 4 major emergent themes from the qualitative analysis of participant beta tests include user interface, data content, usefulness during a patient consult, and potential for a patient-focused decision aid. Most participants supported the idea of a patient-focused decision aid but recommended that it should include more background on HCT and a simplification of medical terminology.ConclusionsWe report the development, acceptability, and usability of a prototype decision support tool app to provide individualized risk and survival estimates to patients interested in HCT in a patient consultation setting. We propose to finalize the tool by validating predictive analytics using a large data set of patients with SCD who have undergone HCT. Such a tool may be useful in promoting physician-patient collaboration in making shared decisions regarding HCT for SCD. Further incorporation of patient-specific measures, including the HCT comorbidity index and the quality of life after transplant, may improve the applicability of the decision support tool in a health care setting.

Social Media platforms are increasing user engagements with their attractive User Interfaces (UI) design playing a crucial role in maintaining user involvement. Use of Emotional Intelligence (EI) into the UI transforming user interaction and engagement pattern significantly. With an emphasis on how design decisions affect user behaviour, this study explores the nuanced function of EIUIs (Emotionally Inteligence User Interfaces) in the context of social media. It specifically draws attention to the dangers of manipulative techniques like "dark patterns"-purposeful interface strategies intended to influence users to take actions they might not have otherwise taken-as well as the possibility of constructive involvement. By investigating these practices, the paper hopes to clarify the wider ethical consequences of interface design as well as the duty of platforms and designers to promote transparency, autonomy, and confidence in digital interactions. This paper aims to contribute to the discourse on user interface design by providing a comprehensive understanding of EIUIs' potential and pitfalls in the context of social media interactions. Additionally, this study focusses on finding information of the pattern of Emotionally Intelligent User Interfaces (EIUI) design to enhance user engagements emotionally and tend to take decisions and the impact of the design pattern in user experience.

Computer users with impaired dexterity often have difficulty accessing small, densely packed user interface elements. Past research in software-based solutions has mainly employed two approaches: modifying the interface and modifying the interaction with the cursor. Each approach, however, has limitations. Modifying the user interface by enlarging interactive elements makes access efficient for simple interfaces but increases the cost of navigation for complex ones by displacing items to screens that require tabs or scrolling to reach. Modifying the interaction with the cursor makes access possible to unmodified interfaces but may perform poorly on densely packed targets or require the user to perform multiple steps. We developed a new approach that combines the strengths of the existing approaches while minimizing their shortcomings, introducing only minimal distortion to the original interface while making access to frequently used parts of the user interface efficient and access to all other parts possible. We instantiated this concept as Adaptive Click-and-Cross, a novel interaction technique. Our user study demonstrates that, for sufficiently complex interfaces, Adaptive Click-and-Cross slightly improves the performance of users with impaired dexterity compared to only modifying the interface or only modifying the cursor.

Current authoring environments provide the possibility of developing user interfaces with limited adaptation capacities. The most widely adopted tools follow the responsive design approach and allow developers to obtain user interfaces that can adapt mainly to the screen size and orientation. We present a solution able to support development of user interfaces able to adapt to the various types of contextual events (that can be related to users, devices, environments, and social relationships), with the added possibility of distributing the user interface elements across multiple devices. The context-dependent behavior is modelled through trigger / action rules, and can even be applied to Web applications that were not originally designed to be context-aware. This paper describes the design and main features of the novel authoring environment and reports on a first user study.

An optical brain-machine interface (O-BMI) system using calcium imaging has various advantages such as high resolution, a comprehensive view of large neural populations, abilities such as long-term stable recording, and applicability to freely behaving animals in neuroscience research. The present study developed an ergonomic user interface (UI) design, based on a use scenario for an O-BMI system that can be used for the acquisition and processing of calcium imaging in freely behaving rodents. The UI design was developed in three steps: (1) identification of design and function requirements of users, (2) establishment of a use scenario, and (3) development of a UI prototype. The UI design requirements were identified by a literature review, a benchmark of existing systems, and a focus group interview with five neuroscience researchers. Then, the use scenario was developed for tasks of data acquisition, feature extraction, and neural decoding for offline and online processing by considering the sequences of operations and needs of users. Lastly, a digital prototype incorporating an information architecture, graphic user interfaces, and simulated functions was fabricated. A usability test was conducted with five neuroscientists (work experience = 3.4 ± 1.1 years) and five ergonomic experts (work experience = 3.6 ± 2.7 years) to compare the digital prototypes with four existing systems (Miniscope, nVista, Mosaic, and Suite2p). The usability testing results showed that the ergonomic UI design was significantly preferred to the UI designs of the existing systems by reducing the task completion time by 10.1% to 70.2% on average, the scan path length by 14.4% to 88.7%, and perceived workload by 12.2% to 37.9%, increasing satisfaction by 11.3% to 74.3% in data acquisition and signal-extraction tasks. The present study demonstrates the significance of the user-centered design approach in the development of a system for neuroscience research. Further research is needed to validate the usability test results of the UI prototype as a corresponding real system is implemented.