Human Requirements Validation for Complex Systems Design

The purpose of this article is to propose a View Sharing Augmented Reality (AR) interface aimed at improving situation awareness (SA) during military operations. Battlefield Augmented Reality System (BARS), a mobile AR system, supports real-time battlefield situation awareness (SA) for mission execution through collaborative information sharing. Video-based View Sharing AR interface that provides the perspective of team members located remotely can enhance team-level SA. However, instances have been identified where providing the perspective of team members moving in directions different from the user’s path can lead to user confusion and have a negative impact on mission execution. In this study, we investigate the influence on SA when presenting the perspective of team members with different path direction through a View Sharing AR interface and to examine the impact of visual AR cues supporting spatial awareness on SA improvement. The study was designed to experience the View Sharing AR interface in a virtual reality (VR) environment using the Unity3D game engine. The experiments involved performing tasks to suppress enemies with three different path direction variances in perspective provision and receiving support from four visual AR cues. SA was measured through SA Global Assessment Technique (SAGAT) and Performance measurement. The research findings indicate that differences in path direction negatively affected SA, with the lowest SA observed when the path difference was 90°. Furthermore, visual AR cues, when providing Tramline cues, demonstrated the highest SA, showing interaction effects with path direction differences.

End user involvement and input into the design and evaluation of information systems has been recognized as being a critical success factor in the adoption of information systems. Nowhere is this need more critical than in the design of health information systems. Consistent with evidence from the general software engineering literature, the degree of user input into design of complex systems has been identified as one of the most important factors in the success or failure of complex information systems. The participatory approach goes beyond user-centered design and co-operative design approaches to include end users as more active participants in design ideas and decision making. Proponents of participatory approaches argue for greater end user participation in both design and evaluative processes. Evidence regarding the effectiveness of increased user involvement in design is explored in this contribution in the context of health IT. The contribution will discuss several approaches to including users in design and evaluation. Challenges in IT evaluation during participatory design will be described and explored along with several case studies.

One of the most critical phases in complex socio-technical system development is the validation of non-functional requirements (NFR). During this phase, system designers need to verify that the proposed system’s NFRs will be satisfied. A special type of NFRs which is often ignored regards the Human Factors (HF) NFRs. These requirements are of vital importance to socio-technical systems since they affect the safety and reliability of human agents within such systems. This paper presents a scenario-based approach for validating HF NFRs using VR CAVE simulation. A case study is used to demonstrate the application of the method in the validation of the situation awareness NFR of an in-vehicle Smart driver assistive technology (SDAT). Such systems aim to alleviate accident risks by improving the driver’s situation awareness by drawing their attention on critical information cues that improve decision making. The assessment of the HF NFR is achieved through an experiment with users in a virtual environment. This work describes and demonstrates a method that utilizes a custom-made, modular 3D simulator that uses a number of hazardous scenarios, for the validation of the HF NFRs of prospective systems.

To assess team situation awareness in an aviation maintenance setting, a methodology was developed for examining situation awareness requirements that incorporates both individual and team situation awareness perspectives. In the present study, inquires were conducted in the field maintenance setting at a major airline. Contextual inquires were combined with a goal directed task analysis to specify the situation awareness requirements involved in each of the interactions (between and within teams) required to perform maintenance tasks. Situation awareness requirements in a team context are discussed along with recommendations for training programs directed at improving situation awareness with and between teams.

Visual signals that integrate situation awareness (SA) information can improve drivers’ performance when assuming control of a partially automated vehicle from a non-driving task. By incorporating information that follows the levels of model of SA, these signals enable drivers to more quickly attain higher levels of SA, potentially bypassing traditional sequential processing stages. We evaluated the effectiveness of such SA-enhanced signals (displayed in the instrument cluster) in a simulated driving study by measuring takeover times and collision occurrences. The results indicate that incorporating SA information about the environment significantly reduces the time needed for drivers to take control of the vehicle and results in fewer collisions. Considering these performance differences suggests insights on optimizing visual cues to quickly restore SA among drivers as they transition from non-driving activities to manual control in partially automated vehicles.

Trust is imperative for safe and effective human-automation relationships, especially in complex systems. Transparency, which is the communication of the automation’s behavior and intent to the operator, can be used to build trust in human-automation teams. The present study investigated the impact of transparency across the Human-Automation Teaming (HAT) lifecycle (pre-, during, and post-task) on human-automation trust, communication, and reliance. Participants engaged in a counter small-unmanned air systems simulation. They were randomly assigned to one of six conditions with different configurations of transparency (or lack thereof) across the lifecycle phases. Overall, we found modest outcomes for the influence of automation transparency across the HAT lifecycle on trust, situation awareness, workload, and task performance. While trust levels did not significantly differ across lifecycle configurations, there were notable impacts on situation awareness, task performance, and workload, particularly when transparency was absent during critical phases such as training and the after-action reviews.

This article describes the evaluation of 6 different computer-based training modules created to teach skills that underlie the development and maintenance of situation awareness, a competency shown to be critical in pilot decision-making performance. The modules targeted basic skills training for low-time pilots (checklist completion, air traffic control comprehension, psychomotor skills) as well as training on higher order cognitive skills (attention sharing, contingency planning) and intensive preflight planning. Overall, the training modules improved participants' performance on these targeted skills. Results also provided promising albeit limited support for the effect of the training modules in improving situation awareness.

The crowdsourcing literature has shown that domain experts are not always the best solvers for complex system design problems. Novices and specialists in adjacent domains can, under certain conditions, provide novel solutions at lower costs. Additionally, the best types of solvers for different sub-problems are dependent on the architecture of complex systems. The assignment of solvers based on the architecture, referred to as Solver-Aware System Architecting (SASA), expands traditional system architecting practices by considering solver characteristics and contractual incentive mechanisms in the design process and aims to improve complex system design and innovation by leveraging the strengths of domain experts, crowds, and specialists for different parts of the problem. Given the complexity of system design problems and the variety of solvers available, it is desirable to have heuristics to guide solver assignment to different problems. Developing effective heuristics for solver assignments in complex system design is challenging due to a large number of possible combinations of problem-solver pairs. To address this challenge, this paper presents a computational approach using a multi-armed bandit (MAB) formulation to generate heuristics for solver assignment. The approach is demonstrated using a simple and idealized problem of golf, which has characteristics similar to design problems, including how the problem is decomposed into sub-problems, and solved by different solvers. The results show that the proposed approach is effective in deriving a rich set of heuristics for the golf problem, and can be extended in the future to more complex systems design problems.

This article provides a concrete example for using and creating a virtual environment for computer-aided design of complex systems. The subject area is design, and the object of research is an aircraft. CAD creates a parameterized 3D model, which is demonstrated and processed on a virtual stage using special equipment. Special equipment such as the HTC Vive Pro virtual reality helmet and the included controllers. The virtual assistant of the designer allows you to create detailed parameterized models of an aircraft, in particular, a transport aircraft based on the An-124 Ruslan, and virtual reality helps the user to learn more about the created project and make the necessary optimization edits directly in VR space. By changing the model in VR space, it will change in the main project. The work is a continuation of the development of a decision-making system, a virtual assistant to the designer: "Robot designer", whose application interface has been defined. The main control menu for the user in virtual space is shown, and the aircraft with the fuselage skin and cockpit removed is also shown. Plans for further use of virtual reality technology in the design of complex systems and training of specialists with the profile of aircraft engineers are presented.

At PODC 2000, the theorem received its first broad audience. Surprisingly for an impossibility result, one important effect has been to free designers to explore a wider range of distributed systems. Designers of wide-area systems, in which network partitions are considered inevitable, know they cannot have both availability and consistency, and thus can now justify weaker consistency. The rise of the NoSQL movement (Not Only SQL) is an expression of this freedom. The choices of how and when to weaken consistency are often the defining characteristics of these systems, with new variations appearing every year. We review a variety of interesting places in the CAP Space as a way to illuminate these issues and their consequences. For example, automatic teller machines (ATMs), which predate the theorem, surprisingly choose availability with weak consistency but with bounded risk. Finally, I explore a few of the options to try to work around the impossible. The most basic is the use of commutative operations, which make it easy to restore consistency after a partition heals. However, even many commutative operations have non-commutative in practice, which means that the may be incorrect or late. Adding the concept of exceptions allows more operations to be considered commutative and simplifies eventual consistency during a partition. Furthermore, we can think of delayed exception handling as - we execute a compensating transaction that restores consistency. Delayed exception handling with compensation appears to be what most real wide-area systems do - inconsistency due to limited communication is treated as an exception and some exceptional action, such as monetary compensation or even legal action, is used to fix it. This approach to wide-area systems puts the emphasis on audit trails and recovery rather than prevention, and implies that we should expand and formalize the role of compensation in the design of complex systems

Design of complex systems in the presence of large uncertainties: A statistical approach

The Defense Advanced Research Projects Agency (DARPA) created the Fast, Adaptable, Next-Generation Ground Vehicle (FANG) program to design and develop a new heavy, amphibious infantry fighting vehicle (IFV). DARPA, through the FANG program, is undertaking a radically novel approach to the design and manufacture of complex defense systems, and is hosting a series of prize-based design competitions. These are called the FANG Challenges, and their products will be progressively more complex vehicle subsystems, culminating in the design of a full IFV. They leverage the META design tools and the VehicleFORGE collaboration environment to significantly change the design experience and widen the aperture for design innovation. The META tool suite is a set of free, open source tools for compositional design synthesis at multiple levels of abstraction, design trade space exploration, metrics assessment, and probabilistic verification of system correctness. VehicleFORGE is an open source development collaboration environment for the creation of large, complex systems by numerous unaffiliated designers with the goal of democratizing the design innovation process by engaging several orders of magnitude more talent than the current industry model. This work summarizes the results from the first FANG challenge through analysis of participants, team structures, and the designs created.

Recent studies in interface-mediated situation awareness claim that an ecological approach to interface design can be a good foundation to support successful decision-making, even in unanticipated situations. It is also claimed to serve well as a teaching tool that can promote a deeper understanding of the work domain dynamics. Although previous studies in pilot terrain and traffic awareness support these claims, the results of pilot-in-the-loop evaluations did not always reveal that t he improved situation awareness and decision-making could be fully attributed to the ecological interfaces. The experiment designs compared them to conventional pilot interfaces, which were not always designed for the same purpose. In this paper an extensive pilot-in-theloop experimental evaluation of an Ecological Synthetic Vision Display, described in the companion paper, will be presented that aims to make fair comparisons with respect to a viable design alternative. Furthermore, it also aims to test the above-mentioned claims. The pilot-in-the-loop experiment, using sixteen glass-cockpit pilots in a fixed-based flight simulator, showed that the ecol ogical overlays indeed improved the overall pilot terrain awareness and decision-making in unanticipated events as compared to a command-based interface counterpart. The utility of the energy angle was found to be most important for recognizing off-normal events. However, for short-term collision avoidance, flight safety in terms of low altitude flying, and pilot workload the results are in favor of the command display. Despite these results, none of the pilots crashed when using the ecological display and the majority of them preferred it.

The sense of touch as a man-machine communication channel can be as acute as the sense of sight and sound. In some scenarios such as those seen in aerobatics, stunt flying, and combat flights, tactile sensors can even outperform the conventional non-contact sensors in terms of situation awareness. Fusion of tactile sensory information with those obtained via sight and sound can avoid diverting the user’s attention away from the operational task at hand as well. In this study, the performance of an operator, to servo control the motion of a 2-dof model helicopter with pitch/yaw maneuverability, subjected to an intuitive body-referenced arrangement of a cluster of vibro-tactile sensors is investigated. A blindfolded operator will then control the helicopter to a safe attraction zone via a joystick based on this tactile sensory information. A fine-tuned local controller would take over for the end-of-motion precise homing. This study can pave the way towards a systematic integration and characterization of tactile sensors in high performance weapon platforms with improved situation awareness in visually awkward maneuvers such as those seen in aerial combat scenarios.

Facing flight centred air traffic control (ATC) and more monitoring tasks at future controller working positions (CWP), it becomes even more important that air traffic controllers (ATCo) always focus their attention at the relevant spots on their human machine interface (HMI). This paper outlines relevant literature about attention and attention guidance (AG) in different domains, explains the concept for an AG prototype for sectorless air traffic management (ATM) and the plan for its validation in the course of Single European Sky ATM Research (PJ.16-04-03, SESAR2020). The AG prototype considers three aspects. First, the desired area of attention: An assistance system calculates where the ATCo should look at depending on input data like radar and flight plan data. Second, an external system relying on eye tracking and user inputs determines where the current ATCo focus is. Third, if the desired and the actual area of attention are not the same, mechanisms to guide the ATCo’s attention will be triggered taking a strategy of escalating visual cues into account. The latter comprises an intelligent display of action indicators with respect to time, position and appearance as well as pre-tactical inattention indicators. The AG prototype aims to increase controller productivity, improve situation awareness as well as reduce workload and stress level in a future flight centred ATC environment.