Basic Research in Human Factors

This paper examines the role of human factors in the design of automobiles. A prime objective of our human factors profession is to improve the design of machines, thereby benefiting users in terms of comfort, convenience, operating speeds, accuracy and safety. Although the purpose of an automotive human factors program may be to achieve all of these objectives by improving vehicle design, the mechanisms for doing so probably cannot be discovered by focusing research attention on the vehicle element of the driver/vehicle/road system. In fact, the nonvehicle parts of this system are probably by far the most productive topics for future human factors research. The abilities of drivers, their limitations, and the tasks imposed upon them by the traffic environment should indicate how vehicles can be designed to best serve the drivers' needs. After twenty years of automotive study, the human factors research community is surprisingly unprepared to participate in vehicle design projects. The vehicle has too often ended up the subject of human factors research and researchers have been faced with the job of finding ways to improve the vehicle or a vehicle component without knowing enough about the intended user or the job the user must perform. The research community has only rudimentary and often incomplete background information about drivers and their traffic environments. The meager data base which is available suggests that traditional empirical approaches for evaluating machine design may be too cumbersome and time consuming to keep pace with other aspects of automotive technological evolution. The tradition of developing alternative versions of hardware and subjecting the alternatives to human performance tests may not be a viable methodology in the future. A look at the total automotive system shows why. Drivers in the United States accumulate about 1.6 trillion miles of travel each year. During the year, a typical driver makes over 60,000 discrete control operations not counting steering wheel movements. The immensity of the automotive system means that very small driver error rates in control usage quickly accumulate into large numbers of error events nationwide. The best information available suggests that the D. S. driving public uses their turn signals 854 billion times a year. This amounts to a nationwide rate of 27,000 times per second. If the generic human error rate in using the turn signal can assumed to be one error per 1000 operations, then turn signal errors are being made at the rate of 27 per second nationwide. Human factors research has tended to avoid error rate as a principal measure of performance in research programs. The reason becomes apparent when the number of tests which must be conducted to detect changes in rare events such as turn signal errors is computed. If two turn signal designs are to be compared and the researcher wants to be able to detect with 95 percent certainty (at the 5% level of significance) that the error rate has been cut in half by one of the two designs, then a large experiment is required. At a base human error rate of 1 per thousand, about 130,000 observations must be conducted to reliably detect the desired reduction in errors. If the base human error rate is only 1 in 100, then only 13,000 observations will be needed. Unfortunately, information on the frequency of driver control usage is sketchy, and data on driver error rates when using controls under the natural loading of the driving task is all but nonexistent. Other measures of human performance, such as speed of operation and accident involvement rates, have limited application in automotive design for reasons that are discussed in this paper. Some of the data bases which have been accumulated for human factors evaluations by Ford Motor Company are described in this paper. It is concluded that, if the human factors profession is to keep pace with automotive technological evolution, more research effort is going to have to be devoted to the study of drivers and driving-environment factors. For the sake of research efficiency, human factors principles and systems models which can be reliably generalized across vehicle designs must be developed. Several systems models that are under development at Ford are briefly described.

<div class="htmlview paragraph">The major goal of Space Human Factors program is to provide the technology which will enable safe and productive human performance throughout and after long duration space flights and planetary/lunar missions. The Human Factors element has three major sub-goals:</div> <div class="htmlview paragraph"> <ol class="list nostyle"> <li class="list-item"> <span class="li-label">1.</span> <div class="htmlview paragraph">Human Performance Models, Data and Tools - provide the basic understanding of human's activities in the physical, mental, and perceptual domains and to identify important lessons learned about human activities from prior space and lunar missions. To this end, Johnson Space Center is using a laser-based human body mapping system to measure different anthropometric sizes and shapes for incorporation into a large database. This database is employed, along with a spatial computer image graphics system, to analyze human motion in zero gravity conditions as the hypothetical, human bodysized figure performs a variety of new procedures in Space Station Freedom or in the Shuttle orbiter. At Ames Research Center, human performance models for computer-aided engineering, a program initially developed for aeronautical applications, are being modified for space applications. The value of the dynamic human performance models lies in the versatility in which various management decisions can be evaluated as to their impact on mission effectiveness, personnel skill and knowledge allocation, workload, workstation and cockpit design, and overall system design.</div> </li> <li class="list-item"> <span class="li-label">2.</span> <div class="htmlview paragraph">Crew Support and Enhancement - address the development of technology and its applications for the evaluation of the crew's living and working activities in the spacecraft and planetary/lunar habitat and on planetary/lunar surfaces. During the year, a number of projects were completed. At Johnson Space Center, new computer graphics were developed, particularly new methods for portraying procedures which reduce human error. Also, advances in information management methods for application to Shuttle maintenance operations were developed at Ames Research Center. Data from Kennedy Space Center operations is being used as a testbed for managing the information processing in maintenance.</div> </li> <li class="list-item"> <span class="li-label">3.</span> <div class="htmlview paragraph">Human-Automation-Robotic Systems - determine the means by which humans and automated or semi-autonomous systems can effectively work together. A major project at Ames Research Center is the development of the Virtual Workstation as a user interface device for exploring lunar or planetary surfaces. The effort this year has been to obtain a Mar-like data base and begin to employ it in simulated missions. At Johnson Space Center, effort was directed toward identifying the best location of visual sensors for teleoperations.</div> </li> </ol> </div>

Human performance plays an important role in system performance and is influenced by a wide set of internal, organizational, and environmental factors. Human–robot systems are being developed for a large number of domains, including in-home care, military deployments, emergency response, industrial robots, and therapeutic applications. Human performance modeling provides the opportunity to investigate multiple system prototypes and theories regarding human performance that can be further evaluated in human-in-the-loop experiments, to quickly adapt to rapidly changing robotic technology, and to represent human behavior in extreme conditions. Interacting with robots also influences human performance, and models of the human–robot system provide an avenue to investigate the impact on human performance that the human–robot teaming elicits. This chapter provides an overview of examples representing common methods for assessing human performance in human–robot systems. The topics are summarized in Table 3.2 and include a description of human performance modeling, a summary of commonly used human performance modeling tools, examples of human performance modeling with a focus on its application in human–robot systems, information regarding the validation of models, and guidelines for implementing human performance modeling techniques for robotics.

We go the HFES Annual Meeting to get an overview of our field, but we usually go to niche sessions that focus on presenting specific experimental results. The goal of this session is to begin to get HFES members to think more strategically about the content of the Annual Meeting. In this session, two accomplished presenters will talk about developments in their respective areas of expertise—what we know, what we don't know, what new developments pertain to other areas of human factors/ergonomics, and where other fields of interest can contribute to medical HF/E and human performance modeling.

The human factors have a significant impact on the flight performance. A computable human performance model (HPM) is proposed in this paper to simulate the operations and performances of a real human pilot. Incorporated with cognitive psychology theory, the proposed HPM attempts to explain the principles and mechanisms of the human factors such as the problem how the human operational time delay and errors were introduced. The proposed model contains a series of methods to expound the change of human time delay and human errors corresponding to information variations. Finally, a simulation experiment is carried out to study the impact of human time delay and decision-making biases on the flight performance.

<div class="section abstract"><div class="htmlview paragraph">Evaluation of integrated human-machine systems depends on having accurate human performance models. However, such models often provide only instantaneous snapshots of cognitive state and fail to account for ongoing dynamics. We argue that generative AI solutions can be used to alleviate this problem. Generative AI tools have been successful when applied to problems that have repeatable structure captured by a low-dimensional lexicon and associated with large amounts of training data. These properties apply to human performance modeling as well. Here, we introduce our Generative Cognitive Modeling Tool, a prototype human performance model developed using strategies from the generative AI community. We demonstrate the utility of our approach using simulated driving data. Our results show that cognitive states associated with driving errors are not randomized events but rather the outcome of continuous dynamics and are predictable up to 25 secs prior to the error event. We also find that the voluntary utilization of autonomous driving aids can be predicted, in part, by the disruption of ongoing dynamics. Overall, this underscores the importance of ongoing dynamics for human performance modeling and establish that generative AI approaches can provide one way to account for such factors.</div></div>

The field of human factors studies the interaction between humans and technological systems in order to optimize human–system performance, research that has traditionally been focused on observation and experiment in laboratory and real-world settings. However, a multidisciplinary subfield of human factors called human performance modeling has recently emerged that involves the development and application of mathematical models, computer simulation techniques, and computational data analysis methods to the study of human performance. This chapter provides an introduction to the use of computational modeling, simulation, and analysis techniques in human factors research, with an eye toward how such methods may be used to optimize human performance in extreme settings.

Verification and validation (V&V) for human performance models (HPMs) can be likened to building a house with no bricks, since it is difficult to obtain metrics to validate a model when the system is still in development. HPMs are effective for performing trade-offs between the human system designs factors including number of operators needed, the role of automated tasks versus operator tasks, and member task responsibilities required to operate a system. On a recent government contract, our team used a human performance model to provide additional analysis beyond traditional trade studies. Our team verified the contractually mandated staff size for using the system. This task demanded that the model have sufficient fidelity to provide information for high confidence staffing decisions. It required a method for verifying and validating the model and its results to ensure that it accurately reflected the real world. The situation caused a dilemma because there was no actual system to gather real data to use to validate the model. It is a challenge to validate human performance models, since they support design decisions prior to system. For example, crew models are typically inform the design, staffing needs, and the requirements for each operator’s user interface prior to development. This paper discusses a successful case study for how our team met the V&V challenges with the US Air Force model accreditation authority and successfully accredited our human performance model with enough fidelity for requirements testing on an Air Force Command and Control program.

Expert system knowledge represents expertise obtained through formal education, training, and/or experience. Formal education provides deep knowledge of a particular domain; experience and training result in heuristic knowledge. A knowledge base defines the range of information and understanding with which the system is capable of dealing; therefore, its information must be structured and filed for ready access. The objective of this symposium is to address the challenges associated with establishment of valid expert system knowledge, specifically, knowledge to be used by expert system shells. As expert system knowledge is obtained, structured, and stored, it is formulated. In this symposium, knowledge formulation is addressed as a three-phase process: knowledge acquisition, the mechanics associated with structuring knowledge, and knowledge porting. Knowledge acquisition is the process of extracting expertise from a domain expert. Expertise may be collected through a series of interviews between the expert and a knowledge engineer or through sessions the expert holds with an automated knowledge acquisition tool. Thus, the ultimate outcome of knowledge acquisition is a collection of raw knowledge data. The following human factors issues become apparent: documenting mental models (where mental models are the expert's conceptualization of a problem), recording cognitive problem-solving strategies, and specifying an appropriate interface between the domain expert and the acquisition methodology. The knowledge structuring process involves the refinement of raw knowledge data, where knowledge is organized and assigned a semantic structure. One issue that must be considered is how to interpret knowledge data such that formal definitions, logical relationships, and facts can be established. Finally, formulation involves knowledge porting, that is, the movement of an expert system shell's knowledge base to various other shells. The outcome of this process is a portable knowledge base, where the challenges lie in maintaining consistent knowledge, understanding the constraints inherent to a shell (the shell's ability to incorporate all relevant knowledge), and designing an acceptable user-expert system interface. The fundamental component of any expert system is its knowledge base. The issues to be presented in this symposium are important because they address three processes that are critical to the development of a knowledge base. In addition to presenting computer science challenges, knowledge base formulation also presents human factors challenges, for example, understanding cognitive problem-solving processes, representing uncertain information, and defining human-expert system interface problems. This symposium will provide a forum for discussion of both types of challenges.

: The Air Force Research Laboratory's Human Performance Model Integration (HPMI) program is exploring the merit and feasibility of combining multiple human performance models and tools to create hybrid models of performance that address application-specific requirements for model fidelity while controlling cost. The first exploration of HPMI feasibility involves development of a human performance model using task network modeling and ACT-R cognitive modeling. Under the Combat Automation Requirements Testbed program, a detailed task network model of strike fighter pilot performance was developed and integrated with a flight simulator. In the initial implementation the task network model managed prioritization of objects and made decisions about the order in which the targets would be examined. This original task network model of prioritization then was replaced with one implemented in ACT-R (Anderson and Lebeire, 1998). The cognitive model provided improved representations of decision-making and memory, as well as confusion and errors committed by humans. We will describe the task domain, model and interface, as well as, an informal verification study that compared the performance of the hybrid model with that of humans.

This panel introduces the current practice of human performance modeling and its applications to Human Factors research and discusses how we can promote modeling education. We argue that there is a necessity for new and more engaging learning tools and environments that can include qualitative and quantitative modeling work in Human Factors curricula to fulfill different levels of approaches in Human Factors research. We hope that this panel can encourage more students and junior researchers to utilize modeling in their work and spark exciting discussions for better modeling practice and education.

Formal analyses using human mental models have been effective in identifying how automation behavior can cause mode confusion and associated human error. However, state-machine-based mental model formalisms are unrealistically precise. To address this, we introduce Fuzzy Mental Models (FMMs). FMMs combine state machines with fuzzy logic to precisely represent vagueness in mental models. We define how FMMs integrate into larger formal system models, where the state of the automation and FMM co-evolve. We identify patterns within a system model that indicate mode confusion. These include new conditions that emerge only through the vagueness captured by FMMs. We also present a model-based analysis capable of discovering the mode confusion patterns in system models containing FMMs. Our approach is demonstrated with two realistic case studies: a modern automobile gear shifter and a cloud storage file system. We conclude by discussing the implications of our contributions and directions for future research.

BackgroundWide-ranging concerns exist regarding the use of black-box modelling methods in sensitive contexts such as healthcare. Despite performance gains and hype, uptake of artificial intelligence (AI) is hindered by these concerns. Explainable AI is thought to help alleviate these concerns. However, existing definitions for explainable are not forming a solid foundation for this work.MethodsWe critique recent reviews on the literature regarding: the agency of an AI within a team; mental models, especially as they apply to healthcare, and the practical aspects of their elicitation; and existing and current definitions of explainability, especially from the perspective of AI researchers. On the basis of this literature, we create a new definition of explainable, and supporting terms, providing definitions that can be objectively evaluated. Finally, we apply the new definition of explainable to three existing models, demonstrating how it can apply to previous research, and providing guidance for future research on the basis of this definition.ResultsExisting definitions of explanation are premised on global applicability and don’t address the question ‘understandable by whom?’. Eliciting mental models can be likened to creating explainable AI if one considers the AI as a member of a team. On this basis, we define explainability in terms of the context of the model, comprising the purpose, audience, and language of the model and explanation. As examples, this definition is applied to regression models, neural nets, and human mental models in operating-room teams.ConclusionsExisting definitions of explanation have limitations for ensuring that the concerns for practical applications are resolved. Defining explainability in terms of the context of their application forces evaluations to be aligned with the practical goals of the model. Further, it will allow researchers to explicitly distinguish between explanations for technical and lay audiences, allowing different evaluations to be applied to each.

The purpose of this workshop is to provide individuals who are considering the use of computer modeling and simulation with an understanding of 1) what simulation is, 2) when it might be useful, and 3) what specific considerations and approaches should be used when using simulation to solve problems. In addition to lectures, attendees will perform a series of hands-on exercises with pre-built Micro Saint computer models. Attendees will use the models to solve real world design problems including 1) evaluating the necessary word recognition rate to make computer speech input more efficient than mouse input and 2) determining the optimal makeup of a team given task demands and function allocation. Also, attendees will receive hands on Micro Saint model building experience through the construction of a simple model of a human driving while using a cell phone. Participants in the workshop do not require any specific educational background. Knowledge of human factors, workplace design, workload analysis and cognitive processing is beneficial though not required. All that is required are basic computer skills with the Windows operating system.

Vehicles with partial automation, forerunners to those with higher levels of automation, are already being deployed by automakers. These current deployments, although incremental, have the potential to disrupt how people interact with vehicles. This chapter reports on a discussion of related issues that was held as part of the Human Factors Breakout session at the 2017 Automated Vehicle Symposium. The session, titled “Automated Vehicle Challenges: How can Human Factors Research Help Inform Designers, Road Users, and Policy Makers?”, included discussions between industry experts and human factors researchers and professionals on immediate human factors issues surrounding deployment of vehicles with Automated Driving Systems (ADS).