Quantitative Assessment of Drone Pilot Performance

This research aims to characterize extended reality flight trainers and to provide a detailed account of the sensors employed to collect data essential for qualitative task performance analysis, with a particular focus on gaze behavior within the extended reality environment. A comparative study was conducted to evaluate the effectiveness of an extended reality environment relative to traditional flight simulators. Eight flight instructor candidates, advanced pilots with comparable flight-hour experience, were divided into four groups based on airplane or helicopter type and cockpit configuration (analog or digital). In the traditional simulator, fixation numbers, dwell time percentages, revisit numbers, and revisit time percentages were recorded, while in the extended reality environment, the following metrics were analyzed: fixation numbers and durations, saccade numbers and durations, smooth pursuits and durations, and number of blinks. These eye-tracking parameters were evaluated alongside flight performance metrics across all trials. Each scenario involved a takeoff and initial climb task within the traffic pattern of a fixed-wing aircraft. Despite the diversity of pilot groups, no statistically significant differences were observed in either flight performance or gaze behavior metrics between the two environments. Moreover, differences identified between certain pilot groups within one scenario were consistently observed in another, indicating the sensitivity of the proposed evaluation procedure. The enhanced realism and validated effectiveness are therefore crucial for establishing standards that support the formal adoption of extended reality technologies in pilot training programs. Integrating this digital space significantly enhances the overall training experience and provides a higher level of simulation fidelity for next-generation cadet training.

Background: The increasing discourse on advanced automation and artificial intelligence (AI) in aviation is fuelled by the pursuit of enhanced safety and greater efficiency. The design of takeover requests should be appropriately contextual, rather than fixed, to support optimal human performance. Aim: This paper focused on adaptive automation in aircraft cockpits. The aim was to explore three factors related to the transition from automated to manual aircraft control, to investigate the impact of cognitive workload of non-flying related tasks (WoNFRTs, i.e. high and low), takeover request modality (TORM, i.e. visual and auditory), role of automation (ROA, i.e. monitoring and best-option), and their interactions on aircraft takeover performance and perceptual responses. Method: A Wizard-of-Oz experiment (within-subjects, N=40) employing a 2 (WoNFRTs) x 2 (TORM) x 2 (ROA) design was conducted. Performance metrics (i.e. task completion time) as well as perceptual metrics (i.e. mental workload, situation awareness, trust, and usability) were collected. Results: This study found that task completion time was significantly improved with: (1) high WoNFRTs and (2) best-option mode. Although not significant, best-option mode improved perceived situation awareness. The combination of ‘auditory + best-option’ performed significantly better than certain combinations on task performance and situation awareness under low WoNFRTs, while no significant difference was found among TORM and ROA combinations under high WoNFRTs. The findings lay a solid foundation for future investigations aimed at a deeper understanding of the relationship between human factors, pilot performance, perceptions, and the design of workload-adaptive takeover requests in aviation.

Contents: B.H. Kantowitz, Series Foreword. Preface. Part I:Introduction. J.M. Koonce, A Historical Overview of Aviation Human Factors. J.E. Deaton, J.G. Morrison, Aviation Research and Development: A Framework for the Effective Practice of Human Factors, or What Your Mentor Never Told You About a Career in Human Factors... D. Meister, Measurement in Aviation Systems. D.W. Abbott, M.A. Wise, J.A. Wise, Underpinnings of System Evaluation. R. Westrum, A.J. Adamski, Organizational Factors Associated With Safety and Mission Success in Aviation Environments. Part II:Human Capabilities and Performance. L. Bainbridge, Processes Underlying Human Performance. R.R. Amalberti, Automation in Aviation: A Human Factors Perspective. C. Prince, E. Salas, Team Processes and Their Training in Aviation. D.E. Maurino, Crew Resource Management: A Time for Reflection. G. Costa, Fatigue and Biological Rhythms. M.R. Endsley, Situation Awareness in Aviation Systems. D.L. Pohlman, J.D. Fletcher, Aviation Personnel Selection and Training. Part III:Aircraft. L. Hitchcock, Pilot Performance. J.M. Reising, K.K. Liggett, R.C. Munns, Controls, Displays, and Workplace Design. W.F. Moroney, B.W. Moroney, Flight Simulation. L.E. Kovarik, R.C. Graeber, P.R. Mitchell, Human Factors Considerations in Aircraft Cabin Design. B.E. Hamilton, Helicopter Human Factors. Part IV:Air Traffic Control. M.S. Nolan, Air Traffic Control. D.J. Garland, E.S. Stein, J.K. Muller, Air Traffic Controller Memory: Capabilities, Limitations, and Volatility. V.D. Hopkin, Air Traffic Control Automation. K. Kerns, Human Factors in Air Traffic Control/Flight Deck Integration: Implications of Data-Link Simulation Research. Part V:Aviation Operations and Design. J.M. Hammer, Human Factors of Functionality and Intelligent Avionics. T.A. Lindholm, Weather Information Presentation. C.G. Drury, Human Factors in Aviation Maintenance. R.J. Lofaro, Human Factors in U.S. Civil Aviation Security. S. Baker, Aviation Incident and Accident Investigation. R.D. Gilson, Forensic Aviation Human Factors [Accident/Incident Analyses for Legal Proceedings].

Human-cockpit interaction is an innovative and promising field of automotive research. Indeed, automakers need to ensure safety and user satisfaction for their cockpit development concepts, if driving and interacting occurs simultaneously. One suggested approach is to evaluate simple cockpit prototypes within virtual test environments. Hybrid prototyping allows a more realistic experience with the prototype in early stages of development. With our research study we focused on important basic parameters within hybrid test environments (e.g. shutter glasses and virtual projected car model) and evaluated their potential of influence. There are no hints to assume that shutter glasses influence user behaviour. Interestingly, we found significant faster task completion times within a virtual projected car model, which indicate that immersive environments increase user performances. In summary, we can suggest hybrid prototyping within immersive test environments for evaluating human-cockpit interactions.

MSIE-Transformer: A novel driving behavior modeling approach for virtual simulation test environment.

Research demonstrated that personality traits of fighter pilots have impact on their performance. The prediction of supersonic fighter pilots performance and professional success can be made by assessment of the levels of personality traits of those individuals that are selected to practice this profession.
 The present research examined if the personality traits (self-efficacy, orderliness, dutifulness and altruism) can predict the level of performance of the fighter pilots on supersonic aircraft, by using the “IPIP NEO” (International Personality Item Pool NEO) based on Big Five model and inter-evaluation method within group of supersonic fighter pilots. The results indicated that the high performance of supersonic aircraft fighter pilots can be predicted by high levels of above-mentioned personality traits and that there is a significant positive relation between those variables. 
 The great need for success, the desire to adapt, to overcome their limitations and to face the difficulties, allow pilots to exercise their profession with all the special requirements of missions and the challenges in air combat.

Objective Use of teleneuropsychological services has greatly increased since the beginning of the COVID-19 pandemic. The present study aimed to create normative data for a neuropsychological test battery of diverse cognitive domains in a Canadian population. Method A sample (n = 291) of adults aged 19 or older completed a comprehensive neuropsychological assessment (i.e. memory, executive function, etc.) via Zoom. Participants included those with a COVID-19 diagnosis (n = 146) and participants who had not contracted COVID-19 (n = 145). Data were stratified by age group as follows: 19–34, 35–49, 50–64, 65–79. Linear bivariate regression in the entire sample and groups stratified by age was used to test the relationship between age and test scores. Test scores were converted to z-scores using the mean and standard deviation for that group, with z-scores then transformed into normative scores for each test. Results Age was a significant predictor of scores for all tests except for FAS, HVLT-R (Retention, Recognition), and Digit Span (Forwards, Backwards). After raw test scores were regressed onto age for each group, age was no longer a significant predictor for most test scores, with exceptions for each age group. Conclusions This study created normative data for a diverse teleneuropsychological test battery in a Canadian population. Standard­ized scores generally fell within the average range, with the exception of TOPF and JLO scores, which may be explained by high educational attainment and virtual testing environment, respectively. The results suggest that the teleneuropsychological testing environment results in similar performance to in-person assessment.

Todays’ product development process is characterized by an increasing use of embedded software solutions integrated into mechatronic products. The development is more and more translocated into a virtual environment. New software methods and tools have to be developed. Industry 4.0 is an approach to highlight the tendency of modern development. Communication between smart products, communication via internet technologies, cyber-physical systems and the Internet of Things are the basis of Industry 4.0. Owing this development, used project management methodologies have to be adjusted. In special the well-known V-Model is now extended to the W-Model to cope with the new requirements like communications between different disciplines. New approaches in virtual development have to be adapted to modern teaching techniques. Therefore a course for first semester mechanical engineering students is conducted by the department of Computer Integrated Design at the Technische Universität Darmstadt. Industry 4.0 fundamentals are taught as well as the development process underlying the so called W-Model. The students will apply this knowledge while they participate in exercises. A web-based tutorial is provided every week with different learning packages. With these learning packages, the students learn to use the project management techniques as well as software development techniques to solve different tasks. Later complex data structures and algorithms can be coded and are applied. The software development techniques, established in development of information technologies, gets more important in mechanical engineering. Therefore the students learn these aspects. Over three months length the students work in groups and use all their skills to realize a bigger software-project — a digital factory. They use a virtual testing environment (ViTMeS 3.0) to develop their solution. The presented ViTMeS 3.0 is a further development of a virtual testing environment used in last year’s team work. Later they can test their code with a real life example. This example, the digital factory, built with LEGO Mindstorms, is an important part of teaching students the foundations of communication and information techniques as well as software development and programming skills. The last step of the team work is the coding of a graphical user interface for appropriate visualization.

In this study we investigate the dependencies between human focus of attention and heart rate variability while performing concentration task in virtual environment, using attention improvement device, and taking into account daily physical activity of the participants. For this purpose, a virtual testing environment – football arena – was developed. A volunteer had to “kick” a ball to the gate. He scores a goal if he has reached high attention focus level. The experiments were performed using NeuroSky device for attention level recording, Polar V800 device for heart rate (RR intervals) recording and ElfEmmit device as a better focus stimulator. At the same time, some virtual environment parameters and human action in it were recorded as well. We have used RMSSD parameter as heart rate variability measure. The results showed some unexpected tendencies. RMSSD parameter was in normal range and outside it for both: volunteers who have regular trainings and who do not have any training at all. A tendency was noticed that ElfEmmit device might increase focus of attention during virtual reality game.

<title>Abstract</title> <p><bold>Background.</bold> Unmanned aerial vehicles (UAV) play a crucial role in modern military and civilian operations, requiring peak cognitive performance from their pilots. Drone pilots often experience attention lapses and performance reductions, which can jeopardize mission success, task performance, and their well-being. Existing research shows that video game players outperform non-gamers in certain cognitive abilities, such as visual tracking, attention, and mental rotation. This study explores how video game experience may shape the skills and workload perception of potential UAV pilots. <bold>Method.</bold> A pilot study was conducted with two groups: gamers (<italic>n</italic> = 22) and non-gamers (<italic>n </italic>= 29). Participants carried out simulated drone flight tasks using a <italic>Phantom 3 </italic>on the <italic>Aerosim RC</italic> simulator. Performance and workload were assessed using the AWT, a test derived from the NASA-TLX. <bold>Results</bold>. Significant differences were found between gamers and non-gamers in both performance and workload. The Mann-Whitney U test revealed that gamers completed the simulated flight tasks in less time (<italic>z =</italic> -4.168, <italic>p <</italic> .01) and made fewer errors (<italic>z =</italic> -4.690, <italic>p <</italic> .01) compared to non-gamers. Additionally, gamers reported significantly lower workload across all variables measured by the AWT.<bold> Discussion.</bold> The findings suggest that video game experience enhances drone piloting performance and reduces perceived workload. These results have implications for refining training and selection protocols for UAV pilots in both military and civilian contexts. Incorporating gaming-based assessments into the training process could provide a promising avenue to enhance UAV operator capabilities and mission success.</p>

Remote ship operation is an emerging technology that presents unique challenges for human operators. The development of robust test methods and standardised assessment procedures is a central basis for the investigation of human factors in the remote control of ships. A key factor for the safety and efficiency of these systems is the assessment of the situation awareness (SA) of remote operators compared with on-site navigators. Systematic analysis of existing measurement methods is required to identify relevant metrics, test set-ups and performance indicators. The focus here is on the evaluation of existing studies that deal with the quantification of human factors in long-distance shipping. Test environments that have already been developed and the measurement methods used will be examined. A comparison of the evaluation methods for remote-controlled systems with established methods for conventional ship navigation provides a basis for assessing the performance and workload of remote operators. Analysing the central challenges of remote control of ships is essential to develop realistic test environments that support future studies on human-machine interaction in remote-control systems. These challenges include limitations in sensory perception such as reduced visual field of view and lack of physical feedback, delays in communication and ergonomic deficits at human-machine interfaces. The possibilities to reproduce these challenges in test scenarios are discussed. A particular focus is on the BEYOND method of the Fraunhofer CML. It enables a systematic evaluation of SA in reproducible scenarios through simulation-based testing and standardised performance metrics. Finally, guidelines for standardised tests in remote-control environments can be derived from the knowledge gained. The development of methodological approaches for a structured evaluation of remotely controlled ship management contributes to creating a reliable basis for safe and efficient operating models.

User proficiency in manual operation of autonomous systems is crucial to the performance of these systems because users are often the final barrier in detecting and correcting abnormal behavior in autonomy. This letter presents a new approach to identifying user proficiency in piloting small unmanned aerial vehicles (sUAVs) by first extracting meaningful features and then using a clustering method to generate ground truth. Pilot performance has been broadly explored in the field of aviation, but not for operation of sUAVs, and both are inherently different due expectations for training, location of user, and subsequent change in user’s point-of-view. We propose a novel, hybrid approach to evaluate UAV pilot performance: combining human-rater data and computational methods that incorporate performance metrics to tune and homogenize the process (of applying controls) and the product (UAV flight path) of piloting sUAVs. The results reveal a spectrum of user skills that designers of these systems need to account for and the ways that users at different skill levels can be expected to respond, informing future autonomy design. In a 20 participant study, users were asked to fly a sUAV along 8 different flight paths of varying difficulty while the flight trajectory and user control inputs were recorded. We utilized unsupervised learning techniques to group pilots into proficiency groups and analyzed the clusters with respect to the features built. We also identified possible factors based on groupings to target training of these users. We validated our approach using new set of data from 12 participants.

The procurement of the National Automated Fingerprint Identification System (NAFIS) involved thorough and rigorous assessment of the system throughout the duration of the development phase of the contract. The system was benchmarked to assess operational accuracy and performance. HCI trials assessed usability and other human computer interaction factors. These were major issues that affected performance and had to be addressed to ensure the system was successfully implemented. Human factors such as the viewing of respondents on screen, interactive encoding, manual verification, search specification, training and the level of user experience of AFIS systems are often subjective, difficult to measure and hard to control in a test environment. These factors contribute to ambivalence in the accuracy data making it difficult to ascertain whether the performance measures for the system are a result of the system’s accuracy or the user’s ability to work with the interface. Much work was done in the design of the tests for NAFIS to mitigate these effects. The benchmark trials and HCI trials complemented each other by providing useful information for measuring the usability, throughput and operational performance of NAFIS but also helped to distinguish areas of the system where human factors affected accuracy and performance.

The take-off and landing of an aircraft is a complex and challenging task for the pilot. Most accidents occur in this phase of flight while flying duration is only 4% of total flying time. Implementation of Head-up displays (HUDs) in the cockpit enhances the pilot’s SA and reduces workload. But presenting large amounts of data on HUD with different display attributes degrades pilot performance, leading to an accident in high workload conditions. This study examines the effect of the sudden change in Symbology and presents large data at a time on pilot performance. It also discussed a model to reduce human errors during take-off and landing modes of operation in low visibility conditions. The work discussed in this manuscript is novel as no experiment has been conducted to observe the human factor’s relation to operational modes and display attributes on see-through displays under low visibility conditions.

This scoping review explores the potential of Artificial Intelligence (AI) to improve pilot training, a critical component in maintaining aviation safety. Traditional training methods, while foundational, are limited by cost, access to real-world scenarios, and adaptability to individual learning needs (AI Labs, n.d.). AI technologies, such as machine learning, computer vision, and natural language processing, offer innovative tools to enhance the cognitive and decision-making capabilities of pilots, enabling personalized learning, real-time feedback, and adaptive simulation environments (Kabashkin et al., 2023; Eddins, 2024). A comprehensive review of academic databases was conducted using specific keywords related to AI and pilot performance. Studies included in this review focused on AI applications that directly impact pilot training efficacy. Data were extracted and qualitatively analyzed to assess the scope and quality of findings across selected literature.Key findings reveal a shift from instructor-led and simulator-based training to more dynamic, AI-enhanced platforms that simulate real-time stressors and decision-making contexts. Additionally, AI supports safety outcomes by powering predictive analytics to identify potential risks before they occur (Mylrea & Robinson, 2023). Despite these advances, challenges remain. Ethical issues such as algorithmic bias and data privacy need to be addressed. Technical limitations, including the demand for high-fidelity simulation and AI reliability, also hinder widespread adoption. Human factors, notably pilot trust in AI systems, significantly affect implementation success (Korentsides et al., 2024). This review concludes that while AI presents transformative opportunities for pilot training, its effective integration requires addressing existing limitations, updating regulatory frameworks, and exploring synergies with technologies like virtual and augmented reality. Future research should emphasize collaborative AI systems and ethical safeguards to ensure both technical effectiveness and pilot acceptance.