Prototyping and Evaluating a Real-Time Neuroadaptive Virtual Reality Flight Training System

The use of virtual reality (VR) for flight simulation, particularly in the earliest stages of pilot training, is gaining attention in both research and industry. The use of the technology for this ab initio training requires suitable consideration of the risks of simulator sickness—risks that are heightened relative to conventional simulators. If simulator sickness results in the development of compensatory skills, or otherwise disrupts the training process, the benefits of the technology may be negated. Enabling the effective integration of VR within flight training requires that, to the extent that simulator sickness is an issue, practical mechanisms are developed to manage the occurrence without disrupting existing training structures. The primary objective of this research is, thus, to evaluate an intervention and a nuisance factor in relation to the reduction of simulator sickness, considering their practicality within existing flight training syllabi. The Total Severity (TS) of the Simulator Sickness Questionnaire (SSQ) was evaluated within a quasi-experimental, non-equivalent pre-test–post-test design, incorporating three groups: a prior flight experience nuisance factor group, a prior personal computer aviation training device (PCATD) exposure intervention group, and a control group with neither prior experience nor prior simulator exposure. The results indicated that the TS was significantly reduced for the prior flight experience nuisance factor (rrb = 0.375), but that the PCATD exposure intervention produced no such reduction (rrb = 0.016). The findings suggest that VR flight simulation is likely best used as a supplemental tool, introduced after initial airborne experience. Notwithstanding this finding, the relatively low median TS scores (<20) for all groups suggest that the technology may still be used with caution earlier in the training process. No other published research has examined this important effect in the context of the new VR situation.

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

Loss of control in-flight (LOC-I) has historically been the leading cause of commercial airline fatalities. In 2009, an investigation of LOC-I accidents revealed four primary causal factors, citing human factors errors as one of the factors. The investigation discovered that one reason for the human errors was due to deficiencies in pilot training from aircraft stalls and upsets. These deficiencies led the FAA and similar international organizations to implement upset prevention and recovery training (UPRT) guidelines for commercial air transport aircraft. The aviation industry uses flight simulators as part of a comprehensive program for pilot training and certification requirements. Although flight simulators can possess a high level of realism, deficiencies remain. Many pilots find simulators to be easier than their real-world counterparts. Even with a high-fidelity FFS, simulator training currently cannot accurately reproduce essential factors that are considered to be within the safety margins of the aircraft, such as roll and pitch rates and g-loading, all of which can lead to high levels of altitude loss, if not managed correctly. This thesis compared upset recovery maneuvers between a simulator and live flight. The study used in the research required the pilots to recover from an upset condition while also performing an additional task of serially adding two numbers. The secondary task provided a physiological baseline cognitive workload measurement that was used to compare the pilots’ increased workload levels during the recovery phase of the flight. The results indicated that the pilots generally performed better during the simulator flights and had less subjective workload and high levels of situation awareness. The physiological-based workload contrasted subjective measures. However, during a live flight upset recovery, pilots may not be able to transfer the skills acquired from a flight simulator and experience higher levels of workload and reduced levels of situation awareness, increasing their potential for an unsuccessful recovery. Suggestions are made that include more use of live aircraft aerobatic training for UPRT and the use of a physiological-based, unbiased workload assessment during pilot training. Additionally, it is suggested using an adaptive training approach to UPRT that incorporates the primary technical flight performance metrics with secondary task performance.

Hearing loss can compromise U.S. Army aviators' performance, safety, and situational awareness, resulting in increasing mental workload and listening effort. This study evaluated simulated hearing loss on performance and cognitive workload among Army aviators. A mixed-effects linear regression study design was used. A total of 21 aviators underwent clinical audiological testing and simulated flight performance assessments. Simulated hearing loss and workload were manipulated to investigate their effects on speech recognition, flight performance, and subjective workload. Flight simulator routes included normal hearing and simulated hearing loss conditions for both high and low workloads. Task load questionnaires were administered for subjective workload assessments and compared across conditions. Speech recognition scores decreased with increasing levels of hearing loss. In-flight speech intelligibility declined in high workload conditions, with a 26% decrease for mild hearing loss and a 40% decrease for severe hearing loss. High workload conditions degraded flight performance and response times to a secondary task which was exacerbated by simulated hearing loss. Workload scores validated increased workload with simulated hearing loss. No significant findings were observed on the hearing assessment. Findings suggest hearing loss negatively impacts speech recognition and flight performance, especially under high workloads. These results support the importance of addressing hearing loss in aviators. Further research is needed to determine if the clinically adapted Modified Rhyme Test can reflect the impact of hearing loss on aviator performance. Noetzel J, Henry P, Mackie R, Cave K, Stefanson JR, Hale JK, Andres K, Jones H. Simulated hearing loss on speech recognition, flight performance, and workload in aviators. Aerosp Med Hum Perform. 2025; 96(4):269-278.

Brain-Computer Interface (BCI) enables users to interact with a computer only through their brain biological signals, without the need to use muscles. BCI is an emerging research area but it is still relatively immature. However, it is important to reflect on the different aspects of the Human-Computer Interaction (HCI) area related to BCIs, considering that BCIs will be part of interactive systems in the near future. BCIs most attend not only to handicapped users, but also healthy ones, improving interaction for end-users. Virtual Reality (VR) is also an important part of interactive systems, and combined with BCI could greatly enhance user interactions, improving the user experience by using brain signals as input with immersive environments as output. This paper addresses only noninvasive BCIs, since this kind of capture is the only one to not present risk to human health. As contributions of this work we highlight the survey of interactive systems based on BCIs focusing on HCI and VR applications, and a discussion on challenges and future of this subject matter.

This paper aims to study the brain-computer interface (BCI) training system for motor imagery (MI) based on virtual reality. In this paper, the MI-BCI software was designed and implemented by using C# and MATLAB mixed programming method, besides, three types of EEG signal pre-processing algorithms, five feature extraction algorithms, and two classification recognition algorithms were integrated to provide off-line analysis, on-line analysis, and adaptive algorithm selection. The results show that by automatically selecting the optimal combination of algorithms for the subjects, the integration degree between the BCI system and the subjects, and the universality of the BCI system are improved. The test objects can all complete the roaming test by constantly adjusting their state of mind under the feedback of virtual reality. The designed training system in this paper can improve the MI ability of subjects.

Researchers in the field of virtual environments (VE), or virtual reality, surround a participant with synthetic stimuli, The flight simulator community, primarily in the U.S. military, has a great deal of experience with aircraft simulations, and VE researchers should be aware of the major results in this field. In this survey of the literature, we have especially focused on military literature that may be hard for traditional academics to locate via the standard journals. One of the authors of this paper is a military helicopter pilot himself, which was quite useful in obtaining access to many of our references. We concentrate on research that produces specific, measured results that apply to VE research. We assume no background other than basic knowledge of computer graphics, and explain simulator terms and concepts as necessary. This paper ends with an annotated bibliography of some harder to find research results in the field of flight simulators: • The effects of display parameters, including field-of-view and scene complexity; • The effect of lag in system response; • The effect of refresh rate in graphics update; • The existing theories on causes of simulator sickness; and • The after-effects of simulator use Many of the results we cite are contradictory. Our global observation is that with flight simulator research, like most human-computer interaction research, there are very few “correct” answers. Almost always, the answer to a specific question depends on the task the user was attempting to perform with the simulator.

Upwards of 60% of preventable aviation accidents are due to human error, suggesting that sufficient training is vital for improved flight safety. The objective of the present work was to inform the use of virtual reality technology as a flight simulation method that supports training of high-risk scenarios and has major cost saving potential compared to traditional flight simulators.

Embodied interfaces are promising for virtual reality (VR) because they can improve immersion and reduce simulator sickness compared to more traditional handheld interfaces (e.g., gamepads). We present a novel embodied interface called the Limbic Chair. The chair is composed of two separate shells that allow the user’s legs to move independently while sitting. We demonstrate the suitability of the Limbic Chair in two VR scenarios: city navigation and flight simulation. We compare the Limbic Chair to a gamepad using performance measures (i.e., time and accuracy), head movements, body sway, and standard questionnaires for measuring presence, usability, workload, and simulator sickness. In the city navigation scenario, the gamepad was associated with better presence, usability, and workload scores. In the flight simulation scenario, the chair was associated with less body sway (i.e., less simulator sickness) and fewer head movements but also slower performance and higher workload. In all other comparisons, the Limbic Chair and gamepad were similar, showing the promise of the Chair for replacing some control functions traditionally executed using handheld devices.

Virtual reality head-mounted displays (HMDs) have recently incorporated eye-tracking into hardware design. For the present study, the Varjo VR-1 and HTC Vive Pro Eye HMDs were used for three eye-tracking scenarios; and immersion, simulator sickness, and visual discomfort questionnaires measured subjective reactions. A lack of research exists comparing HMDs equipped with eye-tracking capabilities, in terms of the subjective measures employed. In this study, the HMDs were investigated using between-subjects and within-subjects designs. For the between-subjects design, the independent variable was the HMD (i.e., Varjo VR-1 or HTC Vive Pro Eye) and the dependent variables were the immersion, simulator sickness, and visual discomfort measurements. For the within-subjects design, simulator sickness, and visual discomfort were evaluated across scenarios. Forty participants were asked to detect and confirm the same target stimulus in each of the three eye-tracking scenarios of increasing ecological complexity. For the results, non-parametric statistics were used, given non-normal data. Since between-groups differences were not found for immersion, simulator sickness, and visual discomfort, recommendations are provided for updating the survey measures. Within-groups differences for both simulator sickness and visual discomfort are explained for plausible cause and reduction methods. Correlation results indicate overlap between visual discomfort and simulator sickness; immersion lacked significant correlations with the other measures. Overall, survey updates and individual differences (e.g., ocular aspects) should be considered for future research and development.KeywordsEye trackingHead-mounted displaysImmersion simulator sicknessTarget detectionVirtual environmentsVirtual realityVisual discomfort

In order to reduce the cost of flight test and enhance the skills of manipulator, a kind of new solution of unmanned aerial vehicle real-time flight simulation and training system based on XNA framework was put forward. the function module and the structure of system were brought forward, and key technologies such as dynamics and motion model, control interface, human-computer interaction function and visual display were discussed and implemented as well as relevant algorithms. In the dynamic model, the standard four element method was used to describe the attitude angle and the differential equations were solved by the fourth-order runge-kutta method, In the visual module, the principle of infinite terrain generate 3D terrain surface and particle system generated some effects such as rain, snow, fog and explosion et al., In the control module, the human-computer interaction function was achieved by Instructor class of the XNA framework interface. Finally, the system development was realized based on Visual Studio environment and C# language, flight training was simulated in this system by manipulating the handle and controlling the interface, and the feasibility and validity were verified.

The design of virtual environments (VEs) can strongly influence users’ emotions. These VEs are also an important aspect of immersive Virtual Reality (VR) exergames – training system that can inspire athletes to train in a highly motivated way and achieve a higher training intensity. VR-based training and rehabilitation systems can increase a user’s motivation to train and to repeat physical exercises. The surrounding VE can potentially predominantly influence users’ motivation and hence potentially even physical performance. Besides providing potentially motivating environments, physical training can be enhanced by gamification. However, it is unclear whether the surrounding VE of a VR-based physical training system influences the effectiveness of gamification. We investigate whether an emotional positive or emotional negative design influences the sport performance and interacts with the positive effects of gamification. In a user study, we immerse participants in VEs following either an emotional positive, neutral, or negative design and measure the duration the participants can hold a static strength-endurance exercise. The study targeted the investigation of the effects of (1) emotional VE design as well as the (2) presence and absence of gamification. We did not observe significant differences in the performance of the participants independent of the conditions of VE design or gamification. Gamification caused a dominating effect on emotion and motivation over the emotional design of the VEs, thus indicating an overall positive impact. The emotional design influenced the participants’ intrinsic motivation but caused mixed results with respect to emotion. Overall, our results indicate the importance of using gamification and support the commonly used emotional positive VEs for physical training. They further indicate that the design space could also include other emotional directions of VE design.

BackgroundFor severely paralyzed people, a brain-computer interface (BCI) provides a way of re-establishing communication. Although subjects with muscular dystrophy (MD) appear to be potential BCI users, the actual long-term effects of BCI use on brain activities in MD subjects have yet to be clarified. To investigate these effects, we followed BCI use by a chronic tetraplegic subject with MD over 5 months. The topographic changes in an electroencephalogram (EEG) after long-term use of the virtual reality (VR)-based BCI were also assessed. Our originally developed BCI system was used to classify an EEG recorded over the sensorimotor cortex in real time and estimate the user's motor intention (MI) in 3 different limb movements: feet, left hand, and right hand. An avatar in the internet-based VR was controlled in accordance with the results of the EEG classification by the BCI. The subject was trained to control his avatar via the BCI by strolling in the VR for 1 hour a day and then continued the same training twice a month at his home.ResultsAfter the training, the error rate of the EEG classification decreased from 40% to 28%. The subject successfully walked around in the VR using only his MI and chatted with other users through a voice-chat function embedded in the internet-based VR. With this improvement in BCI control, event-related desynchronization (ERD) following MI was significantly enhanced (p < 0.01) for feet MI (from -29% to -55%), left-hand MI (from -23% to -42%), and right-hand MI (from -22% to -51%).ConclusionsThese results show that our subject with severe MD was able to learn to control his EEG signal and communicate with other users through use of VR navigation and suggest that an internet-based VR has the potential to provide paralyzed people with the opportunity for easy communication.

Human-machine systems designed for daily activities must balance direct user control with autonomous system behaviour to reduce cognitive effort while maintaining responsiveness. Douglas et al. have demonstrated how shared-autonomy frameworks can enable BCIs to operate across multiple levels of user involvement, from direct neural control to high-level goal specification. Rather than relying solely on low-level EEG commands, the proposed approach integrates intent inference with adaptive robotic planning, allowing multiple users to coordinate multiple robots in functional daily tasks. A key insight from this contribution is that increasing autonomy can significantly reduce user workload while preserving task accuracy and efficiency, supporting more sustainable long-term BCI use in real-world settings. This work highlights a broader shift in BCI design toward collaborative control paradigms, where machines proactively assist users instead of acting only as passive command receivers.Single-modality EEG BCIs are often limited by noise, signal ambiguity, and performance variability across users and environments. Hybrid approaches that fuse EEG with additional sensing modalities offer a promising solution, as demonstrated by Coutray and their team. Coutray et al. have proven how integrating EEG with eye-tracking improves command disambiguation, interaction speed, and overall control reliability, particularly in immersive virtual reality environments. This hybrid design reduces reliance on high-precision EEG decoding alone and enables more natural, intuitive hands-free interaction, expanding BCI applicability beyond clinical contexts into entertainment, accessibility, and smartenvironment control. This contribution reinforces the growing consensus that multimodal BCIs are more scalable, resilient, and user-friendly than EEG-only systems. Robust BCI performance depends not only on decoding algorithms but also on the user's cognitive and neural state. Mohamed et al. have provided evidence that baseline neural oscillations, particularly precue alpha activity, influence event-related desynchronisation (ERD) strength, a core signal used in motorimagery BCIs. A concrete implication of this finding is that BCIs could dynamically adjust classification thresholds, training protocols, or feedback timing based on real-time cognitive state estimates, potentially improving accuracy, consistency, and user learning rates. These insights support the development of state-aware BCIs that adapt decoding strategies in response to moment-to-moment brain dynamics, improving reliability in naturalistic, everyday environments.Beyond EEG decoding alone, combining physiological stress markers and subjective workload measures enables more comprehensive prediction of user performance in demanding tasks. Wei et al. have demonstrated that interpretable machine-learning models can forecast task outcomes while revealing which physiological factors most strongly influence performance. This is particularly relevant for realworld BCI deployment, where fatigue, stress, and cognitive overload can degrade system reliability. A key insight is that interpretable predictive models can support adaptive intervention strategies, such as adjusting task difficulty or providing rest prompts, while maintaining transparency and user trust.Across the contributions, several overarching themes emerge:1. Multimodal integration: Combining EEG with complementary sensors enhances decoding robustness and interaction flexibility. 2. Adaptive autonomy: Systems that balance user intent with automated assistance reduce workload and improve task efficiency. 3. Machine-learning innovation: Deep and interpretable models improve accuracy while supporting transparency and trust. 4. Cognitive context awareness: Accounting for neural and psychological state enables more reliable and personalised BCI performance.Together, these themes suggest a transition from static, single-user BCIs toward adaptive, context-aware, and multi-agent systems designed for long-term everyday use.

This chapter explores the transformative potential of technologies like brain–computer interfaces (BCI) and virtual reality (VR), predicting their profound impact on human experience. It highlights BCI advancements, such as reconstructing music from brain signals and restoring movement in paralysed patients, emphasizing their medical and technological significance. The chapter also touches on commercial interest in BCI, particularly Elon Musk’s Neuralink, and discusses the concept of humans merging with machines, as envisioned by Ray Kurzweil. Regarding VR, the author predicts that future developments will lead people to spend more time in virtual worlds than the physical one, with richer sensory experiences and even interactions with AI representations of the deceased. The chapter concludes by addressing the ethical, privacy, and identity issues these technologies raise, encouraging openness to their potential while remaining mindful of the challenges.