HPMI: Integrating Systems Engineering and Human Performance Models

Providing an authoritative representation of human behavior has recently been recognized as essential for modeling and simulation of system performance. Even earlier; however, the Human Research & Engineering Directorate of the U.S. Army Research Laboratory was working to do just that in a tool called IMPRINT, (the Improved Performance Research Integration Tool). IMPRINT enables a quantitative portrayal of human performance. Task network modeling is implemented with embedded data and sound psychological methods in an easy-to-use interface, all of which has been subjected to intensive verification and validation. IMPRINT is being used today to influence system design and acquisition decisions. Several examples of applications to Army systems are discussed, with implications ranging from detailed equipment design to force-on-force effectiveness. Of course, “more research needs to be done,” but in the meantime, human performance modeling is ready for use.

In this study, task-network models using the Improved Performance Research Integration Tool were built to predict the mental workload and performance of the crew of a conceptual mounted combat vehicle. The modeling project analyzed which crewmember, if any, should be allocated the functions of controlling robotic assets in addition to standard mission functions similar to present tank crew missions. In order to further investigate issues derived from the modeling project and to verify its analytical results, a simulation experiment was conducted to examine the mental workload and performance of the combined position of robotics operator and gunner. The results of this experiment were consistent with the predictions generated by the modeling analysis. The modeling and experimental data have successfully contributed to modifications in the design concept for the modeled platform as well as its crewmember function allocations.

<div class="htmlview paragraph">The major goal of Space Human Factors program is to provide the technology which will enable safe and productive human performance throughout and after long duration space flights and planetary/lunar missions. The Human Factors element has three major sub-goals:</div> <div class="htmlview paragraph"> <ol class="list nostyle"> <li class="list-item"> <span class="li-label">1.</span> <div class="htmlview paragraph">Human Performance Models, Data and Tools - provide the basic understanding of human's activities in the physical, mental, and perceptual domains and to identify important lessons learned about human activities from prior space and lunar missions. To this end, Johnson Space Center is using a laser-based human body mapping system to measure different anthropometric sizes and shapes for incorporation into a large database. This database is employed, along with a spatial computer image graphics system, to analyze human motion in zero gravity conditions as the hypothetical, human bodysized figure performs a variety of new procedures in Space Station Freedom or in the Shuttle orbiter. At Ames Research Center, human performance models for computer-aided engineering, a program initially developed for aeronautical applications, are being modified for space applications. The value of the dynamic human performance models lies in the versatility in which various management decisions can be evaluated as to their impact on mission effectiveness, personnel skill and knowledge allocation, workload, workstation and cockpit design, and overall system design.</div> </li> <li class="list-item"> <span class="li-label">2.</span> <div class="htmlview paragraph">Crew Support and Enhancement - address the development of technology and its applications for the evaluation of the crew's living and working activities in the spacecraft and planetary/lunar habitat and on planetary/lunar surfaces. During the year, a number of projects were completed. At Johnson Space Center, new computer graphics were developed, particularly new methods for portraying procedures which reduce human error. Also, advances in information management methods for application to Shuttle maintenance operations were developed at Ames Research Center. Data from Kennedy Space Center operations is being used as a testbed for managing the information processing in maintenance.</div> </li> <li class="list-item"> <span class="li-label">3.</span> <div class="htmlview paragraph">Human-Automation-Robotic Systems - determine the means by which humans and automated or semi-autonomous systems can effectively work together. A major project at Ames Research Center is the development of the Virtual Workstation as a user interface device for exploring lunar or planetary surfaces. The effort this year has been to obtain a Mar-like data base and begin to employ it in simulated missions. At Johnson Space Center, effort was directed toward identifying the best location of visual sensors for teleoperations.</div> </li> </ol> </div>

<div class="section abstract"><div class="htmlview paragraph">Evaluation of integrated human-machine systems depends on having accurate human performance models. However, such models often provide only instantaneous snapshots of cognitive state and fail to account for ongoing dynamics. We argue that generative AI solutions can be used to alleviate this problem. Generative AI tools have been successful when applied to problems that have repeatable structure captured by a low-dimensional lexicon and associated with large amounts of training data. These properties apply to human performance modeling as well. Here, we introduce our Generative Cognitive Modeling Tool, a prototype human performance model developed using strategies from the generative AI community. We demonstrate the utility of our approach using simulated driving data. Our results show that cognitive states associated with driving errors are not randomized events but rather the outcome of continuous dynamics and are predictable up to 25 secs prior to the error event. We also find that the voluntary utilization of autonomous driving aids can be predicted, in part, by the disruption of ongoing dynamics. Overall, this underscores the importance of ongoing dynamics for human performance modeling and establish that generative AI approaches can provide one way to account for such factors.</div></div>

Human performance plays an important role in system performance and is influenced by a wide set of internal, organizational, and environmental factors. Human–robot systems are being developed for a large number of domains, including in-home care, military deployments, emergency response, industrial robots, and therapeutic applications. Human performance modeling provides the opportunity to investigate multiple system prototypes and theories regarding human performance that can be further evaluated in human-in-the-loop experiments, to quickly adapt to rapidly changing robotic technology, and to represent human behavior in extreme conditions. Interacting with robots also influences human performance, and models of the human–robot system provide an avenue to investigate the impact on human performance that the human–robot teaming elicits. This chapter provides an overview of examples representing common methods for assessing human performance in human–robot systems. The topics are summarized in Table 3.2 and include a description of human performance modeling, a summary of commonly used human performance modeling tools, examples of human performance modeling with a focus on its application in human–robot systems, information regarding the validation of models, and guidelines for implementing human performance modeling techniques for robotics.

Verification and validation (V&V) for human performance models (HPMs) can be likened to building a house with no bricks, since it is difficult to obtain metrics to validate a model when the system is still in development. HPMs are effective for performing trade-offs between the human system designs factors including number of operators needed, the role of automated tasks versus operator tasks, and member task responsibilities required to operate a system. On a recent government contract, our team used a human performance model to provide additional analysis beyond traditional trade studies. Our team verified the contractually mandated staff size for using the system. This task demanded that the model have sufficient fidelity to provide information for high confidence staffing decisions. It required a method for verifying and validating the model and its results to ensure that it accurately reflected the real world. The situation caused a dilemma because there was no actual system to gather real data to use to validate the model. It is a challenge to validate human performance models, since they support design decisions prior to system. For example, crew models are typically inform the design, staffing needs, and the requirements for each operator’s user interface prior to development. This paper discusses a successful case study for how our team met the V&V challenges with the US Air Force model accreditation authority and successfully accredited our human performance model with enough fidelity for requirements testing on an Air Force Command and Control program.

This paper discusses the utility of task-network modeling for quantifying human performance variability in microgravity. The data are gathered for: (1) improving current methodologies for assessing human performance and workload in the operational space environment; (2) developing tools for assessing alternative system designs; and (3) developing an integrated set of methodologies for the evaluation of performance degradation during extended duration spaceflight. The evaluation entailed an analysis of the Remote Manipulator System payload-grapple task performed on many shuttle missions. Task-network modeling can be used as a tool for assessing and enhancing human performance in man-machine systems, particularly for modeling long-duration manned spaceflight. Task-network modeling can be directed toward improving system efficiency by increasing the understanding of basic capabilities of the human component in the system and the factors that influence these capabilities.

Factors related to workers and tasks have a combined effect on system performance. Management tends to make effective decisions by understanding the dynamics of the workers' performance over time, affected by their time-varying learning, fatigue, and stress levels. Such managerial decisions could improve system productivity and ensure workforce safety. Despite the importance of the problem, the literature did not focus on the performance modelling or task planning framework that includes all the mentioned factors, i.e., learning, fatigue, and stress. This dissertation includes three main contributions with different performance modelling and task allocation/scheduling planning that help managers plan mission operations, characterized as uncertain, dynamic, and time-sensitive. The first contribution (Chapter 3) is a mathematical model that modifies a popular learning curve model from the literature by making its learning exponent dependent on the fatigue level. The results of applying this model to a data set (vs. the other available ones in the literature,) showed an outperformance in terms of efficiency and balance criteria.

Assessing decision heuristics using machine learning

: Six studies are in various stages of completion in the CORE program of the Committee on Human Factors. Preliminary drafts of reports on integrated ergonomic modeling, human performance models, mental models, and multicolored displays have been prepared and are currently in the prepublication review process. In addition, reports of the workshop on distributed decision making and the seminar on expert systems, both convened in late 1986, are being drafted. The technical plans have been completed for a panel meeting on human factors for an aging population. The committee is in the process of specifying its research agenda for the next study period. Keywords: Human factors; National Research Council; Human performance models; Multicolored displays; Mental models; Integrated ergonomic models; Distributed decision making; Expert systems; Aging; Productivity.

We go the HFES Annual Meeting to get an overview of our field, but we usually go to niche sessions that focus on presenting specific experimental results. The goal of this session is to begin to get HFES members to think more strategically about the content of the Annual Meeting. In this session, two accomplished presenters will talk about developments in their respective areas of expertise—what we know, what we don't know, what new developments pertain to other areas of human factors/ergonomics, and where other fields of interest can contribute to medical HF/E and human performance modeling.

Cognitive human performance models have enjoyed a rich history in human-computer interaction but have yet to make a widespread impact in system design, possibly because they are difficult to construct. We employed user-centered design techniques to develop a new tool that is easier to use than previous methods or tools. CogTool combines a familiar method of prototyping, modeling by demonstration, and the ACT-R cognitive architecture to enable user interface designers to make valid human performance models with little effort.

Human performance modeling (HPM) has been effective for determining crew designs. Crew design includes determining the number of operators needed, the role of automation, and member task responsibilities required to operate a system. Crew models are built for target systems under development to better inform the design and the needs of each operator’s user interface. This is a Verification and Validation (V&V) challenge—it is difficult to obtain metrics to validate a model when the completed system does not yet exist for obtaining sample measures. This paper addresses the challenge with an implementation of the V&V approach applying it to an HPM that successfully received V&V Accreditation for a large United States Air Force (USAF) command and control system. This paper then compares the results of the validated model with NASA TLX workload metrics and discusses the V&V effectiveness. Finally, it recommends better methods for V&V for human performance models.

The human factors have a significant impact on the flight performance. A computable human performance model (HPM) is proposed in this paper to simulate the operations and performances of a real human pilot. Incorporated with cognitive psychology theory, the proposed HPM attempts to explain the principles and mechanisms of the human factors such as the problem how the human operational time delay and errors were introduced. The proposed model contains a series of methods to expound the change of human time delay and human errors corresponding to information variations. Finally, a simulation experiment is carried out to study the impact of human time delay and decision-making biases on the flight performance.

Collision warning systems offer a promising approach to mitigate rear-end collisions, but substantial uncertainty exists regarding the joint performance of the driver and the collision warning algorithms. A simple deterministic model of driver performance was used to examine kinematics-based and perceptual-based rear-end collision avoidance algorithms over a range of collision situations, algorithm parameters, and assumptions regarding driver performance. The results show that the assumptions concerning driver reaction times have important consequences for algorithm performance, with underestimates dramatically undermining the safety benefit of the warning. Additionally, under some circumstances, when drivers rely on the warning algorithms, larger headways can result in more severe collisions. This reflects the nonlinear interaction among the collision situation, the algorithm, and driver response that should not be attributed to the complexities of driver behavior but to the kinematics of the situation. Comparisons made with experimental data demonstrate that a simple human performance model can capture important elements of system performance and complement expensive human-in-the-loop experiments. Actual or potential applications of this research include selection of an appropriate algorithm, more accurate specification of algorithm parameters, and guidance for future experiments.