User Interface Validation using Mode Confusion Detection

This article focuses on a major human factors–related issue that includes the undeniable role of cultural factors and cockpit automation and their serious impact on flight crew performance, communication, and aviation safety. The report concentrates on the flight crew performance of the Boeing 777–Asiana Airlines Flight 214 accident, by exploring issues concerning mode confusion and autothrottle systems. It also further reviews the vital role of cultural factors in aviation safety and provides a brief overview of past, related accidents. Automation progressions have been created in an attempt to design an error-free flight deck. However, to do that, the pilot must still thoroughly understand every component of the flight deck – most importantly, the automation. Otherwise, if pilots are not completely competent in terms of their automation, the slightest errors can lead to fatal accidents. As seen in the case of Asiana Flight 214, even though engineering designs and pilot training have greatly evolved over the years, there are many cultural, design, and communication factors that affect pilot performance. It is concluded that aviation systems designers, in cooperation with pilots and regulatory bodies, should lead the strategic effort of systematically addressing the serious issues of cockpit automation, human factors, and cultural issues, including their interactions, which will certainly lead to better solutions for safer flights.

Situational Awareness (SAW) is a concept widely used in areas that require critical decision making, such as in the field of emergency management. SAW is related to the level of perception and understanding that an individual has about real events occurring in complex scenarios, which must be managed by critical systems. Such critical systems require specialized user interfaces (UI) to give operators a dynamic understanding of what is happening in the environment. A challenging issue in the design of SAW-oriented interfaces is to determine how the human-computer interface process can be constructed for SAW enrichment, considering environments with heterogeneous data sources, limitations in data quality, and ever-changing situations. The problem increases when information is subject to uncertainties, which may compromise the situation analysis process. In addition, humans make decisions based on their own understanding of the event, which allied with experience and knowledge can be valuable assets to be used to process situational information about emergencies for the acquisition of SAW. The objective of this work is to demonstrate how to include a SAI-oriented UI in the process of evaluating emergency situations and to present the development of a UI that promotes the management of situational information of emergencies to promote the acquisition of SAW. The results present a specified routine for employing specialized UIs in SAW as part of a situation assessment process, which supports a strong integration between the human operator and other phases of the process, such as quality assessment, data fusion and information visualization , As well as a prototype interface that meets the process. A case study with a critical scenario of a theft event is also presented to demonstrate the applicability of the proposed approach

Hybrid dynamical systems are those with interaction between continuous and discrete dynamics. For the analysis and control of such systems concepts and theories from either the continuous or the discrete domain are typically readapted. In this thesis the ideas from perturbation theory are readapted for approximating a hybrid system using a continuous one. To this purpose, hybrid systems that possess a two-time scale property, i.e. discrete states evolving in a fast time-scale and continuous states in a slow time-scale, are considered. Then, as in singular perturbation or averaging methods, the system is approximated by a slow continuous time system. Since the hybrid nature of the process is removed by averaging, such a procedure is referred to as dehybridization in this thesis. It is seen that fast transitions required for dehybridization correspond to fast switching in all but one of the discrete states (modes). Here, the notion of dominant mode is defined and the maximum time interval spent in the non-dominant modes is considered as the 'small' parameter which determines the quality of approximation. It is shown that in a finite time interval, the solutions of the hybrid model and the continuous averaged one stay 'close' such that the error between them goes to zero as the 'small' parameter goes to zero. To utilize the ideas of dehybridization for control purposes, a cascade control design scheme is proposed, where the inner-loop artificially creates the two-time scale behavior, while the outer-loop exponentially stabilizes the approximate continuous system. It is shown that if the origin is a common equilibrium point for all modes, then for sufficiently small values of the 'small' parameter, exponential stability of the hybrid model can be guaranteed. However, it is shown that if the origin is not an equilibrium point for some modes, then the trajectories of the hybrid model are ultimately bounded, the bound being a function of the 'small' parameter. The analysis approach used here defines the hybrid system as a perturbation of the averaged one and works along the lines of robust stability. The key technical diffierence is that though the norm of the perturbation is not small, the norm of its time integral is small. This thesis was motivated by the stick-slip drive, a friction-based micropositioning setup, which operates in two distinct modes 'stick' and 'slip'. It consists of two masses which stick together when the interfacial force is less than the Coulomb frictional force, and slips otherwise. The proposed methodology is illustrated through simulation and experimental results on the stick-slip drive.

This paper presents a set of situation awareness (SA) requirements for an operator who monitors critical infrastructure (CI). The requirements consist of pieces of information that the operator needs in order to be successful in their work. The purpose of this research was to define a common requirement base that can be used when designing a CI monitoring system or a user interface to support SA. The requirements can also be used during system or user interface evaluation, and as a guide for what aspects to emphasize when training new CI monitoring operators. To create the SA requirements, goal-directed task analysis (GDTA) was conducted. For GDTA, nine interview sessions were held during the research. For a clear understanding of a CI monitoring operator's work, all interviewees were subject matter experts (SMEs) and had extensive experience in CI monitoring. Before the interviews, a day-long observation session was conducted to gather initial input for the GDTA goal hierarchy and the SA requirements. GDTA identified three goals an operator must achieve in order to be successful in their work, and they were used to define the final SA requirements. As a result, a hierarchy diagram was constructed that includes three goals: monitoring, analysis and internal communication, and external communication. The SA requirements for a CI monitoring operator include information regarding ongoing incidents in the environment and the state of systems and services in the operator's organization.

Hybrid system consists of discrete and continuous state. Fault diagnosis and prediction of hybrid system is a new and critical issue in hybrid system study. Utilizing the advantage of particle filter, which can estimate the discrete and continuous states simultaneously, a new method for fault diagnosing and predicting of hybrid system is proposed. After giving the design steps of particle filter based fault diagnosis and prediction, we discuss the application of the method to the mobile robot fault diagnosis and prediction system. The simulation results show the effectiveness of the proposed approach.

This paper presents a preliminary work of a simulation study for the evaluation of human-machine team (HMT) performance with assistance of a sentinel system under cyberattacks. Sentinel system is an intelligent agent of cyberattack-detection kit, whose user interface (UI) was designed and implemented as a proof-of-concept in the simulation. The goal for this sentinel system is to improve the pilots’ Situation Awareness (SA) of the system vulnerability and resiliency, such that the HMT performance can be improved against disruptive events. Based on the literature of cyberattacks on unmanned aerial vehicle (UAV) control systems, realistic mission operation and cyberattack scenarios were identified and implemented on a simulated UAV ground control station (GCS). In the follow-up experiment, up to twenty-four Air Force pilots will be instructed to supervise the UAVs under specific mission scenarios, and to respond for system recovery solutions generated on Sentinel UI after detection of cyberattacks. Understanding the interactive behaviors of the pilot and Sentinel under mission contexts and cyberattacks is expected to help improve HMT performance.

Currently, digital situational awareness systems are popular in complex infrastructure construction projects. These systems monitor and assess environmental events, progress, resource availability, risks, and other project decision-making variables to support decision-making. However, little is known about how infrastructure construction (IC) professionals who utilize situational awareness systems perceive how they support or hinder situational management. The purpose of this exploratory research is to study, in depth, the relevance, challenges, and adoption of situational management in IC projects using digital systems. The data were collected via semistructured interviews with 21 IC managers and situational awareness management experts from 11 companies involved in railway projects. The main findings indicate that problem-solving improved with situational management in general, especially with digital situational awareness systems. Seizing the possibilities for transparency that accompany digital situational awareness systems helped in discussing emerging problems and making project choices. Expectations about the realism of such expectations were easier to align with historical event data. On the other hand, the informants reported difficulty in motivating contractors to collect situational data in digital form, possibly because of a lack of understanding about the purpose of data collection, the manual nature of data collection, the perceived excessiveness of data collection, or the manual transfer of collected data into digital form. For these reasons, the informants reported limited faith in these systems. A perceived drawback of situational management, whether supported by digital situational awareness systems or not, was its lack of applicability to the realities of a construction site. Systems were designed for project management needs but not tailored to the needs of construction projects. The interviewees’ statements indicate that maintaining situational awareness requires active interaction and constant checking of the provided information, even requiring pressure on the contractors providing the information. This study highlights the need for practical human approaches to effectively use digital situational awareness technologies and situational management in IC.

As agricultural machines become more complex, it is increasingly critical that special attention be directed to the design of the user interface to ensure that the operator will have an adequate understanding of the status of the machine at all times. A user-centred design focus was employed to develop two conceptual designs (UCD1 & UCD2) for a user interface for an agricultural air seeder. The two concepts were compared against an existing user interface (baseline condition) using the metrics of situation awareness (Situation Awareness Global Assessment Technique), mental workload (Integrated Workload Scale), reaction time, and subjective feedback. There were no statistically significant differences among the three user interfaces based on the metric of situation awareness; however, UCD2 was deemed to be significantly better than either UCD1 or the baseline interface on the basis of mental workload, reaction time and subjective feedback. The research has demonstrated that a user-centred design focus will generate a better user interface for an agricultural machine.

Meteorological factors have a profound impact on aircraft operation and safety. Weather-related incidents accounted for 8% of all accident reports, according to data from the U.S. Federal Transportation Safety Commission’s 2009–2018 U.S. aviation accident/quasi-accident survey. The tendency of lightning strikes in domestic airports and surrounding areas is increasing year by year, but unlike major overseas airports, there is no official lightning warning system other than the radio wave system in case of lightning strikes from the Korea Meteorological Administration. Since each operator issues its own alerts based on the Significant Meteorological Information report, small agencies such as low-cost airlines have limited capabilities to mitigate lightning damage. This study aims to support the safety and smooth air traffic control of ground operators and aircraft through the introduction of lightning warning systems and equipment in Korea to protect ground workers and aircraft safety.

The integration of semi-autonomous systems in public transport introduces new design challenges related to user experience, human–machine interaction, and situational awareness. This research, conducted within the Master’s program in Advanced Sustainable Design at the University of Florence in collaboration with an industry leader in autonomous tram technology, explores how user interfaces (UIs) can effectively mediate between automation and human control in complex operational contexts. Using the semi-autonomous tram line of Florence as a case study, the project investigates present issues and future scenarios for interface evolution in public transportation.Students were divided into project groups and engaged in a comprehensive research process combining design thinking and human-centered design approaches. The methodology included a review of the scientific literature, benchmarking of international case studies, and extensive on-field investigation. Empirical data were collected through direct observation, interviews with drivers, and anonymous questionnaires aimed at identifying recurring challenges, user needs, and perceptions of trust and safety in interactions with semi-autonomous systems. This multi-layered analysis enabled a critical comparison between theoretical frameworks and operational realities, revealing gaps and opportunities for UI innovation. The collected evidence was organized into categories of disruptions frequently observed in semi-autonomous scenarios, including issues related to attention management, information overload, and the adaptation of human skills to automated contexts.A parallel line of research explored the evolving role of the driver, focusing on ergonomic factors, cognitive load, stress management, and the need for continuous monitoring of attention and physical condition. Findings demonstrate that the effectiveness of semi-autonomous mobility systems depends largely on the quality of human–machine interaction and the system’s ability to support the operator during high-complexity events. Multimodal feedback—visual, auditory, and tactile—proved to be essential for maintaining situational awareness and reducing reaction time.Building on these insights, the project proposed several future-oriented interface solutions. These included adaptive dashboards capable of dynamically reorganizing information, multimodal alert systems designed to mitigate cognitive overload, and AI-driven virtual assistants to support drivers in diagnosing anomalies, prioritizing warnings, and managing emergency procedures. These speculative scenarios position UIs as cooperative partners that can enhance transparency, reliability, and trust while ensuring meaningful human oversight.Looking ahead, the study highlights the potential of emerging technologies such as Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR) to improve information delivery and skill development. AR overlays within the driver’s field of view could present contextual information—track status, obstacle alerts, route variations—without increasing distraction, while immersive technologies could support training and simulation.Ultimately, this research demonstrates the value of design-driven methodologies in shaping next-generation UIs for public transport systems. By integrating human factors, emerging technologies, and speculative design approaches, the project outlines how adaptive, multimodal, and intelligent interfaces can enhance safety, efficiency, and comfort while guiding the cultural and professional transition toward autonomous mobility.

Remote ship operations are becoming a practical reality, driving the need for well-designed user interfaces (UIs) in remote operation centres (ROCs). These UIs are complex, requiring the integration of physical and digital elements to ensure seamless operations and situational awareness across land- and sea-based contexts. However, the maritime industry currently lacks dedicated design guidelines and precedents for ROCs. This chapter presents the ROC architecture framework, a user-centred approach comprising four interconnected components: (1) a workstation framework for modular, scalable physical console design; (2) a workplace integration framework for harmonising multi-screen set-ups and digital UIs; (3) a remote collaboration and multi-user framework addressing coordination among distributed operators; and (4) a video overlay framework using augmented reality to enhance live video feeds for decision support. Central to this work is the OpenBridge design system, which has supported rapid prototyping and iterative feedback across real-world ROC design cases. The study highlights the importance of using a design system and an open innovation methodology to achieve harmonised and adaptable UIs for current and future remote maritime operations.

It takes movie camera operators years of professional training to follow an object in an aesthetically pleasing way, both by classical means (boom, slider) and by drones. As this complex task requires a high workload and situation awareness while controlling the camera, an uncluttered and efficient user interface (UI) is preferred. The emergence of mobile devices and motion control devices incorporating automation made touch-based UIs attractive to operators. Much work has already been done on UI adaptation strategies. However, little work is trying to solve the problem of combining manual control and automation within a UI. Especially with a central premise of minimising occlusion and visual clutter in a cinematic context. We, therefore, conducted a first user study (N=15) to evaluate different design alternatives regarding occlusion and preference. Afterwards, we created a functional prototype of the most promising design. To further reduce the occlusion we applied a progressive reduction adaption strategy. We evaluated the influence of different reduction levels on workload, control, creativity support and precision in a second user study (N=24). While we could reduce the clutter, due to our design decisions we found no negative effects affecting the measured variables.

We define situation awareness (SA) as adaptive, externally directed consciousness. This definition dispels the artificial and contentious division evident in the literature, according to which SA is either exclusively knowledge or exclusively process. This misdirected rivalry has more to do with general perspectives on the study of human behavior than with SA itself. Through defining SA as an aspect of consciousness, we hope to clarify two key issues. (1) The source of goals with respect to SA is a normative arbiter in the task environment; that is, the behavior that SA generates must be directed at an external goal. (2) SA is the invariant at the core of the agent's perception-action cycle that supports skilled performance; that is, relationships among factors or dimensions in the environment determine what the agent must know and do to achieve the goals specified by the external arbiter. We introduce a construct we call the risk space to represent the invariant relations in the environment that enable the agent to adapt to novel situations and to attain prespecified goals. We articulate this concept of a risk space through use of a specific example in commercial aircraft operations. The risk space structures information about the physical airspace in a manner that captures the momentary knowledge that drives action and that satisfies the goals and performance criteria for safe and efficient flight. We note that the risk space may be generalized to many different means of navigation.

Chevron owns and operates the largest corporate non-commercial aircraft fleet in the world. This paper describes Chevron's aircraft operations and how it supports a large exploration and production operation in the Gulf of Mexico. Discussion focuses on Chevron's highly qualified pilots and mechanics and how they achieve and maintain their technical abilities. The Maintenance Department is described in detail, including procedures for revamping new aircraft to Chevron's standards before they are available for use. Discussion also provides a detailed explanation of the avionics group and their part in outfitting and maintaining a sophisticated flight tracking (Flite-Trak) and Loran system used for navigation. This paper also describes Chevron's own internal safety precautions that may be unique to its operations and have proven to aid in the Company's overall safety program and related accomplishments. The results of Chevron's Aircraft Operations cannot be measured in safety statistics alone. Passenger confidence and employee involvement are two main elements that contribute to Aircraft Operations' success. Chevron aircraft operates in a safe and cost efficient manner. Many American Petroleum Institute (API) and Federal Aviation Administration (FAA) awards have been earned for outstanding safety milestones. It is only through the dedicated efforts of all the employees of Aircraft Operations that such an outstanding organization exists. The credit for the exemplary safety record of this organization truly belongs to the employees.

In this work, a novel computational approach for the dynamic adaptation of User Interfaces (UIs) is proposed, which aims at enhancing the Situational Awareness (SA) of users by leveraging the current context and providing the most useful information, in an optimal and efficient manner. By combining Ontology modeling and reasoning with Combinatorial Optimization, the system decides <italic>what</italic> information to present, <italic>when</italic> to present it, <italic>where</italic> to visualize it in the display - and <italic>how</italic>, taking into consideration contextual factors as well as placement constraints. The main objective of the proposed approach is to optimize the SA associated with the displayed UI <italic>at run-time</italic>, while avoiding information overload and induced stress. In the context of this work, we have deployed our computational approach to the use case of an Augmented Reality (AR) system for Law Enforcement Agents (LEAs). To explore the benefits and limitations of the developed system, two evaluations have been conducted. The first one was an expert-based evaluation with LEAs and User Experience (UX) experts, assessing the appropriateness of the system&#x2019;s decisions. The second one was a user-based evaluation involving LEAs from different agencies, estimating the SA, the mental workload and the overall UX associated with the system, through an AR simulation. The results indicate that the system enhances SA, and while not imposing workload, it provides an overall positive UX.