Mental workload is reflected in driver behaviour, physiology, eye movements and prefrontal cortex activation

Cultural influences on the measurement of subjective mental workload

Driving in monotonous road environments may cause drivers to become bored and impair their alertness. This inattention may increase the risk of road accidents. Drivers usually respond to visual signals on the road. A large body of research has shown that the level of a driver’s visual attention is closely related to traffic accidents. Visual attention is often studied by tracking drivers’ eye movements. However, few studies to date have explored eye-movement patterns in different road environments such as open roads and tunnel expressways. This dissertation aims to address that research gap by studying drivers’ eye movements and visual characteristics in different driving conditions in real and simulated environments. In the pilot study, drivers’ eye movement patterns were recorded during high-speed driving in tunnels and on expressways in Singapore. Twenty-two drivers participated in the study; they drove a total of 55 km, which included a 9-km section of tunnel. The results of the study showed that drivers experienced longer fixation duration when driving in tunnels than on the open road but experienced a higher number of fixations, higher instantaneous velocity, acceleration, and dispersion on open roads than in tunnels. These findings suggest that drivers’ eye movements are more concentrated and have longer fixation durations in tunnels than in open road driving, indicating that drivers have a greater mental workload when driving in tunnels. However, testing drivers in a real environment has limitations. For example, weather and road characteristics cannot be manipulated. To address this limitation, Experiment 1 was conducted in a driving simulator using virtual reality simulations in UC-win/road. Experiment 1 sought to investigate drivers’ eye movements in various driving environments. Eighteen drivers participated in the experiment and drove on normal roads, expressways, and through underground tunnels in both sunny and rainy weather. The comparisons of their eye-movement patterns (e.g., saccadic parameters and pupil size) in the simulated environments were consistent with those in real open-road environments and tunnels in sunny conditions. The findings suggest that the eye-movement patterns of drivers in a simulator are comparable to those in real environments, indicating its ecological validity. Previous literature has suggested a relationship between eye movement patterns and mental workload. To verify that relationship, mental workloads in Experiments 2 and 3 were directly measured using questionnaires. Dynamic and static hazardous scenarios were simulated to induce different levels of mental workload. Experiment 2 investigated eye movements, driving performance, and mental workload of drivers who encountered static and dynamic obstacles while manual driving. The results from 23 drivers found that their mental workload in dynamic conditions was higher than in static hazardous conditions; it was also higher before encountering hazards than afterwards. The results indicated that pupil size and saccade numbers are effective indicators of mental workload. Experiment 3 was conducted under conditionally automated driving conditions, where participants were involved in tasks related to or not related to driving and requested to resume control manually when their vehicle was in semi-automated driving mode. Drivers’ eye movements and mental workloads were measured when they were driving, especially during the transitions from semi-automated to manual driving. Similar to Experiment 2, the results from 32 drivers found that mental workload was higher for dynamic hazards than for static hazards and higher before hazards were encountered than after. The results also indicated that pupil size is the only effective indicator of mental workload. Taken together, the results of these experiments reveal that pupil size is the most reliable way to measure drivers’ mental workload. Furthermore, in line with the load theory of attention and De Waard (1996)’s theory of workload and performance, it can be concluded that mental workload (indicated by pupil size) and visual workload (by saccadic dispersion) are the key determinants of driving performance and that drivers can only respond to emergencies during periods of medium levels of mental and visual workload (not underload or overload) with available attentional capacity. The findings from this thesis on eye movement and mental workload may be used to monitor drivers’ real-time attention and workload and thus provide guidance for the design of safe-driving autonomous vehicles that consider drivers’ capacity for information processing when driving. Knowledge about the limitations and capacity of human drivers in tandem with automated systems may also help improve user acceptance of autonomous vehicles in the future.

Unmanned Aerial Vehicles (UAVs) have created safety problems for the publics. Assessments of the mental workload for UAV operations are essential to realize the causes of UAV accidents. To test the following hypotheses: i. mission difficulty in UAV operation affects both subjective and objective measures of mental workload; ii. mission difficulty affects number of failures in UAV operation. Fourteen male adults participated as UAV operators after attending a UAV training course. They performed four flight missions of different difficulty levels. During their flight missions, their heart rate and inter-beat interval (IBI) were collected. Upon completing each flight mission, the participants gave subjective ratings of mental workload using three commonly adopted assessment tools. The time of flight and number of failures in flight operations were also recorded. The results showed that mission difficulty affected the scores of all three assessment tools significantly. Mission difficulty also affected number of failures and IBI significantly. The scores of the three assessment tools were highly correlated (ρ= 0.7 to 0.83, p < 0.001) with one another. The results of the three subjective ratings were also consistent with that of the IBI data. High mental workload in UAV operation could lead to poor flight performance.

Stroke patients often exhibit difficulties performing a cognitive task while walking, defined as a dual task (DT). Their prefrontal cortex (PFC) activity is higher in DT than in single task (ST). The effects of an increasing load on PFC activity during DT in subacute stroke patients remains unexplored. Our objective was to assess the effects of N-back tasks (low/high load) on cerebral activity, gait parameters, and cognitive performances. Eleven subacute stroke patients (days post-stroke 45.8 ± 31.6) participated in this pilot study (71.4 ± 10 years, BMI 26.7 ± 4.8 kg.m−2, Barthel index 81.8 ± 11.0). Patients completed a STwalk, and 4 conditions with 1-back (low load) and 2-back (high load): STlow, SThigh, DTlow, and DThigh. Overground walking was performed at a comfortable pace and -N-back conditions were carried out verbally. Both gait (speed, stride variability) and cognitive (rate of correct answers) performances were recorded. Changes in PFC oxyhemoglobin (ΔO2Hb) and deoxyhemoglobin (ΔHHb) were measured by functional near infrared spectroscopy (fNIRS). Results showed an increase of ΔO2Hb while walking, which was not augmented by cognitive loads in DT. Walking speed was reduced by low and high cognitive loads in DT compared to STwalk (P < 0.05), but was not different between DTlow and DThigh. Cognitive performances were negatively impacted by both walking (P < 0.05) and cognitive load (between “low” and “high,” P < 0.001). These data highlight a “ceiling” effect in ΔO2Hb levels while walking, leaving no available resources for simultaneous cognitive tasks, during the early recovery period following stroke. In these patients, cognitive, but not motor, performances declined with a higher cognitive load.

Effective assessment of mental workload (MWL) is crucial for reducing human errors during commissioning operations in nuclear power plants (NPPs), but remains a significant challenge. This study investigated the sensitivity of multimodal physiological measures to MWL changes during commissioning operations, and evaluated their effectiveness in MWL modeling using machine learning techniques. Physiological signals were collected during operators’ commissioning operations with varying difficulty levels, and used to build uni-modal, dual-modal and multi-modal MWL assessment models. The results revealed several key physiological measures that were sensitive to MWL changes. Eye movement-based models outperformed unimodal and multimodal models based on EEG and EDA measures for MWL assessment. A random forest model integrating eye movement and EDA measures achieved the best performance. However, combining multiple physiological modalities did not always enhance model performance. The findings provide insights into how different physiological signals can be leveraged to achieve effective MWL assessment for NPP commissioning operations.

Dyre, Warren, and Garness (1996) found that heading performance during active control differed qualitatively from heading performance during passive viewing, though it was unclear exactly why. Our study examined the relationships between field of view, mental workload, and heading perception. We assessed heading performance during active control and passive viewing with a 30° field of view (Dyre et al. used 90°). Subjective mental workload ratings were collected to assess its relationship to active and passive perception of heading. We found results contrary to those of Dyre et al. (1996): heading perception was less accurate during active control than passive viewing and the relationship between simulated velocity and heading performance was reversed. Further, subjective mental workload was higher during active control than during passive viewing. These results suggest that field of view is an important optical variable mediating heading perception during active control.

Driving simulators provide a closely controlled environment for examining links between human physiology and behavior. Physiologic measurements in different driving tasks can assess body temperature, heart rate, respiratory rate, galvanic skin response, electroencephalographic activity, electromyography activity, gastric activity, eyelid closure (as an index of alertness and arousal), and eye movements (as an index of cognitive processing). Synchronous measures of driver physiology and performance in driving simulator scenarios can illuminate relationships between cognitive deficits and performance errors in drivers who lack awareness of self-impairment and environmental hazards. Demonstrations include falling asleep at the controls in drivers affected by cognitive overload, fatigue, prescription medicine or drug effects, or mental decline from brain disease.

Marksmanship is a fundamental skill for all servicemembers. However, the underlying neurophysiological differences in performance among marksmen remain unclear. Incorporating neurophysiological tracking such as functional near-infrared spectroscopy (fNIRS) may identify performance-based differences in prefrontal cortex (PFC) activation during dynamic marksmanship scenarios. This study examined cognitive load within the PFC during a simulated dynamic marksmanship scenario in non-proficient and proficient marksmen. Twenty-four participants (12 men, 12 women) wore an fNIRS device over their forehead during a simulated stationary pistol marksmanship task (to determine proficiency status) and a dynamic shoot/no-shoot course of fire (COF) (to assess cognitive load). Relative concentrations of oxyhemoglobin (HbO2 ), deoxygenated hemoglobin (deoxygenated Hb), and total hemoglobin (total Hb) were tracked to quantify PFC activation differences in twelve proficient (=80% hit percentage) and twelve non-proficient (<80% hit percentage) marksmen. No difference in completion time was observed between groups during the dynamic COF (p=.34). However, non-proficient marksmen showed 26.3% higher HbO2 (p=.02) and 42.1% higher total Hb (p<.01) in the PFC compared to proficient marksmen. Tracking PFC hemodynamic activity identified proficiency-based differences in cognitive load during a dynamic COF. Applying fNIRS during marksmanship-related tasks may be useful in developing stress resilience and mission readiness for servicemembers.

A comprehensive decoding of cognitive load

Mental workload is closely related to changes in performance during a task. For example, performance declines when the mental workload is too high or too low. Also, numerous factors can influence mental workload making it difficult to measure precisely. There have been several operational definitions of mental workload using psychophysiological measures. However, they are cumbersome limiting their application in natural situations. Here, we present a handgrip force sensor designed to be used as a tool for measuring the mental workload of a task. The system is based on force-sensitive resistors placed on the hand fingers. Preliminary experimental results showed that our system can identify accurately the changes in mental workload by association with handgrip force patterns.KeywordsMental workloadHandgrip forceNASA TLX

Brain-computer interface (BCI) based on rapid serial visual presentation (RSVP) is an efficient information detection technology by detecting event related brain response evoked by target stimuli. In the protocol design of the RSVP-BCI a range of parameters could influence the task difficulty, which may result in the changes of mental workload for subjects. This paper focused on the presentation rate in the RSVP paradigm aiming to investigate its influence on mental workload, and the separability of brain states during RSVP tasks with different setup of presentation rate. 64-channel Electroencephalographic (EEG) data were recorded during RSVP tasks with three levels of presentation rate in ten healthy subjects. The results show that different presentation rates indeed contribute to significant differences on mental workload revealed by one-way repeated measures analysis of variance (ANOVA) on z-scored RSME. Higher presentation rate results in the significant decrease on both behavioral and single-trial recognition performance of target images. Classification results on three levels of mental workload show that the mean accuracy reaches 65.5% and the highest accuracy reaches 88.3%. This work implies that mental workload induced by different presentation rates during RSVP tasks could be accurately recognized, and provides a possible method to monitor the mental workload in the application areas of RSVP-BCI.

Exercise interventions, postural control, and prefrontal cortex activation in older adults

Robotics holds the potential to streamline the execution of repetitive and dangerous tasks, which are difficult or impossible for a human operator. However, in complex scenarios, such as nuclear waste management or disaster response, full automation often proves unfeasible due to the diverse and intricate nature of tasks, coupled with the unpredictable hazards, and is typically prevented by stringent regulatory frameworks. Consequently, the predominant approach to managing activities in such settings remains human teleoperation. Teleoperation can be demanding, especially in high-stress situations, and involves a complex blend of both cognitive and physical workload. We present an experiment to explore a range of physiological and performance-related metrics for workload assessment during robotic teleoperation. Thirty-five participants performed a teleoperation task, during which we manipulated cognitive and physical workload conditions. We recorded multiple metrics, including brain activity using functional Near-Infrared Spectroscopy, galvanic skin responses, cardiovascular responses, subjective workload ratings, task and robot performance data. Our results suggest that robotic teleoperation performance may be the most robust metric for distinguishing between different levels of workload experienced during teleoperation, with most physiological measures becoming insignificant to distinguish high cognitive workload.

N-back is a working memory (WM) task to study mental workload on the prefrontal cortex (PFC). We assume that the subject’s performance and changes in mental workload over time depends on the length of the experiment. The performance of the participant can change positively due to the participant’s learning process or negatively because of objective mental fatigue and/or sleepiness. In this pilot study, we examined the PFC activation of 23 healthy subjects while they performed an N-back task with two different levels of task difficulty (2-, and 3-back). The hemodynamic responses were analyzed along with the behavioral data (correct answers). A comparison was done between the hemodynamic activation and behavioral data between the two different task levels and between the beginning and end of the 3-back task. Our results show that there is a significant difference between the two task levels, which is due to the difference in task complication. In addition, a significant difference was seen between the beginning and end of the 3-back task in both behavioral data and hemodynamics due to the subject’s learning process throughout the experiment.

Short-term cardiovascular measures for driver support: Increasing sensitivity for detecting changes in mental workload