A comprehensive decoding of cognitive load

Objective: A set of 4 driving related tasks were used to evaluate the potential for a modified Detection Response Task (DRT) to simultaneously measure visual and cognitive task demands. Background: The accurate assessment of cognitive and visual tasks demands in driving has become increasingly important. As of yet, no simple, cost effective approach has been found to measure visual demands in complex, multimodal tasks. Methods: Two experiments are presented which evaluate an extension of the standard DRT methodology. The discriminate sensitivity of the experiments is tested using an integrated testing configuration, which systematically increased visual demand across four conditions. Results: Results suggest that the standard DRT configurations are highly tuned to selectively evaluate cognitive demand but that a variant of the system may be able to simultaneously evaluate changes in both visual and cognitive task demands. Conclusions: These data suggest that the simple, rapid, and reliable assessment of both visual and cognitive task demands is possible, even in highly fluid systems with non-constant visual task requirements.

Wrist-worn wearables, with sensors to measure physiological responses such as heart rate variability (HRV) and electrodermal activity (EDA), have been increasing and have the potential to be used for continuous monitoring. These devices have been used to detect responses in workers’ physical and cognitive demands; however, the accuracy of wrist wearables for distinguishing these demands is unknown, especially since many every day and work activities frequently require motion. The objective of this study was to evaluate the effectiveness of a wrist-worn wearable in measuring physiological changes during different demand conditions. Participants (n = 20 college students) completed a multi-factor laboratory study that considered task (cognitive/physical), difficulty (easy/hard), and motion (motion/no motion). N-back tasks and stationary bike tasks were used to represent cognitive and physical demands, respectively. Metrics of HRV and EDA were measured using reference-standard devices and a validated wrist-wearable. Significant differences between task, motion, and difficulty were observed from HRV measurements from the reference-standard and wrist-worn devices. Wrist wearables are sensitive to detecting workplace demands and may be used as an alternative to reference-standard sensors for continuous health and activity monitoring for worker health and wellness. Findings in this study can provide guidelines on task and conditions that affect the use and interpretation of wrist-worn devices for measuring cognitive and physical demands in healthcare systems. HIGHLIGHTS Applications for noninvasive, wrist-worn sensors can be used for continuous health and exercise monitoring HRV and EDA metrics obtained from wrist-worn device are sensitivity in detecting changes in task, difficulty, and motion HRV metric from wrist-worn device had agreement with reference-standard device Wrist-wearables has potential for ubiquitous health monitoring of individuals

Towards a general theory of driver behaviour

The association between physiological and eye-tracking metrics and cognitive load in drivers: A meta-analysis

Background: Studying changes in brain activity among table tennis players during tasks of varying difficulty can provide insights into the adaptability and flexibility of the expert brain as well as the formation of cognitive dominance effects. Methods: In this study, national-level table tennis players were recruited to investigate the dynamic processes of brain activity during spatial cognitive tasks of differing difficulty levels. Using event-related potential techniques and functional connectivity analyses, we aimed to determine how the dominant hemisphere for spatial cognition modulates cognitive processing under varying task demands. Results: The P3 wave amplitude in the right hemisphere was consistently greater than that in the left hemisphere regardless of task difficulty. Although the amplitude of the P3 component in the left hemisphere did not significantly differ between low- and high-difficulty tasks, that in the right hemisphere exhibited a significant difference; the amplitude was higher in high-difficulty than in low-difficulty tasks. Functional connectivity analysis further indicated that the brain demonstrated more pronounced synergistic activity in the theta frequency band during high-difficulty tasks. Conclusion: These findings suggest that when task difficulty increases, the brain tends to preferentially activate regions directly related to the task rather than adopting a strategy of unified whole-brain mobilisation. This resource allocation may represent an optimisation mechanism that minimises cognitive waste by intensifying activity in brain regions directly involved in meeting task demands.

Working memory (WM) shows pronounced age-related decline. Functional magnetic resonance imaging (fMRI) studies have revealed age differences in task-related brain activation. Evidence based primarily on episodic memory studies suggests that brain activation patterns can be modulated by task difficulty in both younger and older adults. In most fMRI aging studies on WM, however, performance level has not been considered, so that age differences in activation patterns are confounded with age differences in performance level. Here, we address this issue by comparing younger and older low and high performers in an event-related fMRI study. Thirty younger (20-30 years) and 30 older (60-70 years) healthy adults were tested with a spatial WM task with three load levels. A region-of-interest analysis revealed marked differences in the activation patterns between high and low performers in both age groups. Critically, among the older adults, a more "youth-like" load-dependent modulation of the blood oxygen level-dependent signal was associated with higher levels of spatial WM performance. These findings underscore the need of taking performance level into account when studying changes in functional brain activation patterns from early to late adulthood.

Researchers implementing non-invasive neuroimaging have reported distinct load-dependent brain activity patterns in older adults compared with younger adults. Although findings are mixed, these age-related patterns are often associated with compensatory mechanisms of cognitive decline even in the absence of direct comparisons between brain activity and cognitive performance. This study investigated the effects of cognitive load on brain-behavior relationships in younger and older adults using a data-driven, multivariate partial least squares (PLS) analysis of functional near-infrared spectroscopy (fNIRS) data. We measured bilateral prefrontal brain activity in 31 older and 27 younger adults while they performed single and dual 2-back tasks. Behavioral PLS analysis was used to determine relationships between performance metrics (reaction time and error rate) and brain oxygenation (HbO) and deoxygenation (HbR) patterns across groups and task loads. Results revealed significant age-group differences in brain-behavior relationships. In younger adults, increased brain activity (i.e., increased HbO and decreased HbR) was associated with faster reaction times and better accuracy in the single task, indicating sufficient neural capacity. Conversely, older adults showed a negative correlation between HbR and error rates in the single task; however, in the dual task, they demonstrated a positive relationship between HbO and performance, indicative of compensatory mechanisms under the higher cognitive load. Overall, older adults' showed relationships with either HbR or HbO, but not both, indicating that the robustness of the relationship between brain activity and behavior varies across task load conditions. Our PLS approach revealed distinct load-dependent brain activity between age groups, providing further insight into neurocognitive aging patterns, such as compensatory mechanisms, by emphasizing the variability and complexity of brain-behavior relationships. Our findings also highlight the importance of considering task complexity and cognitive demands in interpreting age-related brain activity patterns.

Evaluation of physiological responses to mental workload in n-back and arithmetic tasks

The open and generative nature of multimedia learning environments tends to cause cognitive overload in learners, and cognitive load is difficult for researchers to observe objectively because of its implicit and complex nature. Event-related potentials (ERP), a method of studying potential changes associated with specific events or stimuli by recording the electroencephalogram (EEG), has become an important method of measuring cognitive load in cognitive psychology. Although many studies have relied on ERP output measurements to compare different levels of cognitive load in multimedia learning, the results of the effect of cognitive load on ERP have been inconsistent. In this study, we used a meta-analysis of evidence-based research to quantitatively analyze 17 experimental studies to quantitatively evaluate which ERP component (amplitude) is most sensitive to cognitive load. Forty five effect sizes from 26 studies involving 360 participants were calculated. (1) The results of the studies analyzed in subgroups indicated high level effect sizes for P300 and P200 (2) Analyses of moderating variables for signal acquisition did not find that different methods of signal acquisition had a significant effect on the measurement of cognitive load (3) Analyses of moderating variables for task design found that a task system with feedback was more convenient for the measurement of cognitive load, and that designing for 3 levels of cognitive load was more convenient for the measurement of cognitive load than for 2 levels of cognitive load. (4) Analyses of continuous moderating variables for subject characteristics did not find significant effects of age, gender, or sample size on the results.

Event Abstract Back to Event Demonstrating brain-level interactions between working memory load and driving demand level using fNIRS Jochem W. Rieger1*, Jakob Scheunemann1, Klas Ihme2, Frank Köster2, Meike Jipp2 and Anirudh Unni1 1 University of Oldenburg, Department of Psychology, Germany 2 Deutsches Zentrum für Luft- und Raumfahrt (DLR), Institute of Transportation Systems, Germany Introduction and Goal The concept of mental workload is of particular importance in human factors and neuroergonomics because mental workload levels that saturate processing capacities will likely decrease performance and cognitive flexibility. Driving, like many other dynamic control tasks, concurrently imposes demands on multiple cognitive resources. In order to assess driver’s cognitive capacity utilization, we need to distinguish loads imposed on different cognitive resources and to characterize their interactions. It is currently not clearly understood how different cognitive task demands interact at the brain level and whether the neuronal markers that have been attributed to certain cognitive resources remain discriminative and predictive of cognitive states when humans operate in complex natural task environments such as driving. Hence, our goal is to investigate how changes in cognitive task demands related to increased lateral control in driving, a visuospatial control task, interacts with working memory load (WML) at the brain level and to predict the driving demand level from multichannel functional near infrared spectroscopy (fNIRS) deoxyhemoglobin (HbR) brain activation measurements. Methods We collected data from participants driving on a highway with moderate concurrent traffic and low or high levels of lateral control demands (driving outside and within construction sites with a narrower lane width) in the realistic Virtual-Reality lab at DLR, Braunschweig (Fischer et al., 2014). In order to manipulate WML levels, participants (n=15) performed an integrated speed regulation task in which they had to remember sequences of speed of varying length. The task resembles a number span task and required the adjustment of the speed when a new speed sign occurred but according to the sign shown n-signs back and to remember the new sign. We introduced five different WML levels (i.e. 0-back to 4-back) and new speed signs appeared every 20 seconds (for details see Unni et al. 2017). We recorded almost whole head fNIRS measurements at a rate close to 2Hz using two 16-channel NIRScout systems (NIRx Medizintechnik GmbH, Germany) during the 60 minutes of the experiment. We then used all fNIRS channels in a multivariate logistic ridge regression classifier with a 5-fold cross-validation for temporally resolved prediction of driving demand level (i.e. whether they drove within or outside the construction site). To gain insights into specific brain areas associated with driving demand level, we performed a univariate group-level analysis where we calculated the prediction model weights from a 5-fold cross-validated univariate logistic ridge regression of channel-wise fNIRS HbR separately for each WML level. Results On average over participants, we correctly predicted the driving demand (lateral control) level in 74.88% of the timepoints in an out-of-sample cross-validation of classifiers trained on the fNIRS data separately for each WML level. Mean accuracy scores significantly varied across WML levels (range 62% to 85% correct; F(4,70) = 7.32, p < .001) with better predictions for intermediate WML levels (1-back to 3-back). Training one classifier on fNIRS data across all WML levels severely degraded prediction performance (mean accuracy of 46%). These results indicate that cognitive resources recruited by lateral lane control, a visuospatial control task, interact with resources recruited by working memory at the brain level. In order to better understand the nature of these interactions, we analyzed how predictivity of brain regions for driving demands develops over WML levels. Figure 1B depicts group-level brain maps of classification separability from the univariate logistic ridge regression model. Activation changes in the parietal and lateral dorsal frontal areas allow for prediction of increased lateral control demand. The frontal areas (putative BA9/46/10) are known to be involved in tactical planning and execution of behavior and the parietal in visuomotor control. The maps show increasing predictivity of these areas with increasing WML levels up to WML-level 2. At WML level 3 and in particular level 4, the discriminative patterns diminish, indicating an interaction between WML and driving demands. We compared these brain maps to the results from Unni et al. 2017, who showed that it was possible to predict WML independent of lateral controls demands from the fNIRS data (average correlation between predicted and induced WML r = 0.61). Figure 1A shows brain areas that predict WML over different lateral control demand levels. In particular, the inferior frontal gyrus (IFG) and the temporo-occipital areas are predictive to the current WML level. The comparison of the anatomical locations of predictive maxima in Figures 1A (marked by white shapes) and 1B suggests only partial overlap between the brain resources predictive of the different task demands. Discussion We demonstrated fNIRS brain level indicators for interaction between WML and driving complexity. The prediction of driving demands from fNIRS data was possible for each WML level separately but dropped to guessing level when we tried to predict driving demands across WML levels. Conversely, we demonstrated the continuous time-resolved predictions of WML over different levels of driving demands in an earlier analysis performed on the same data (Unni et al. 2017). These results indicate an asymmetric pattern of interaction between tasks at the brain level. Moreover, despite the functional interaction, the predictive brain activation patterns show little overlap between the resources predictive for the different tasks. Interpreted in the framework of a multi-resource theory (Wickens, 2002), our brain recordings indicate that working memory and visuomotor coordination in vehicle control recruit different resources and more so with increasing task demand. Moreover, the two tasks appear to interact at a common, task unspecific cognitive resource at the brain level. The changing pattern of driving demand related brain areas across WML levels could indicate potential changes in the multitasking strategy with high levels of WML demand. Figure Caption Figure 1. (A) Weighted averaged group-level univariate correlation HbR brain maps showing brain areas sensitive to changes in WML independent of driving complexity. White shapes mark WML prediction maxima in all maps. (B) Weighted mean of channel-wise predictivity (Tjur R-squared coefficients) for driving demands at the different WML levels. Data for the two analyses were recorded in the same session with concurrent manipulation of WML and lateral control demands. Figure 1 Acknowledgements This work was funded by a grant of the Volkswagen Foundations to the Center for Critical Systems Engineering, a DFG-grant RI1511/2-1 to JWR, and KO 1990/5-1 to FK. References Fischer, M., Richter, A., Schindler, J., Plättner, J., Temme, G., Kelsch, J., & et al. (2014). Modular and Scalable Driving Simulator Hardware and Software for the Development of Future Driver Assistence and Automation Systems. In New Developments in Driving Simulation Design and Experiments: Driving Simulation Conference 2014. Paris, France. Retrieved from http://elib.dlr.de/90638/ Unni, A., Ihme, K., Jipp, M., & Rieger, J. W. (2017). Assessing the driver’s current level of working memory load with high density functional near-infrared spectroscopy: A realistic driving simulator study. Frontiers in Human Neuroscience, 11. https://doi.org/10.3389/fnhum.2017.00167 Wickens, C. D. (2002). Multiple resources and performance prediction, 3(2), 159–177. https://doi.org/10.1080/14639220210123806 Keywords: fNIRS, driving simulator, driving demands, lateral control demands, working memory, visuospatial attention, visuomotor control, Working memory load, Task interaction, task interaction at brain level, parietal cortex, inferior frontal cortex, occitpito temporal cortex Conference: 2nd International Neuroergonomics Conference, Philadelphia, PA, United States, 27 Jun - 29 Jun, 2018. Presentation Type: Oral Presentation Topic: Neuroergonomics Citation: Rieger JW, Scheunemann J, Ihme K, Köster F, Jipp M and Unni A (2019). Demonstrating brain-level interactions between working memory load and driving demand level using fNIRS. Conference Abstract: 2nd International Neuroergonomics Conference. doi: 10.3389/conf.fnhum.2018.227.00141 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 27 Mar 2018; Published Online: 27 Sep 2019. * Correspondence: Prof. Jochem W Rieger, University of Oldenburg, Department of Psychology, Oldenburg, Lower Saxony, 26111, Germany, jochem.rieger@uni-oldenburg.de Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Jochem W Rieger Jakob Scheunemann Klas Ihme Frank Köster Meike Jipp Anirudh Unni Google Jochem W Rieger Jakob Scheunemann Klas Ihme Frank Köster Meike Jipp Anirudh Unni Google Scholar Jochem W Rieger Jakob Scheunemann Klas Ihme Frank Köster Meike Jipp Anirudh Unni PubMed Jochem W Rieger Jakob Scheunemann Klas Ihme Frank Köster Meike Jipp Anirudh Unni Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Communicating and interacting with others is an essential part of our daily routines as humans. Performing these actions appropriately requires the ability to identify, extract, and process salient social cues from the environment. The subsequent application of such knowledge is important for inferring and predicting the behavior of other people. The eyes and brain must work together to fixate and process only the most critical social signals within a scene while passing over and / or completely ignoring other aspects of the scene. While brain activation to isolated presentations of objects and people presentations have been characterized, information about the brain's activation patterns to more comprehensive scenes containing multiple categories of information is limited. Furthermore, little is known about how different interpretations of a scene might alter how that scene is viewed or how the brain responds to that scene. Therefore, the studies presented herein used a combination of infrared eye tracking and functional magnetic resonance imaging techniques to investigate the eye movement and brain activation patterns to socially- and non-socially-relevant interpretations of the same set of complex stimuli. Eye tracking data showed that each gaze pattern was consistent with viewing and attending to only one category of information (people or objects) despite both categories being present in all images. Functional magnetic resonance imaging revealed that a region of the right superior temporal sulcus was selectively activated by the social condition compared to the non-social condition, an area known for its role in social tasks. Brain activation in response to the non-social condition was located in many of the same regions associated with the recognition and processing of visual objects presented in isolation. Taken together, these results demonstrate that in healthy adults, eye movement and brain activation patterns to identical scenes change markedly as a function of attentional focus and interpretation intention. Utilizing realistic and complex stimuli to study the eye gaze and neural activation patterns associated with processing social versus non-social information in the healthy brain is an important step towards understanding the deficits present in individuals with social cognition disorders like autism and schizophrenia.

The aim of this study was to investigate how alcohol intoxication at two blood alcohol concentrations (BAC) affected neuronal activation during increasing levels of cognitive load. For this purpose we used functional magnetic resonance imaging (fMRI) together with a working memory n-back paradigm with three levels of difficulty. Twenty-five healthy male participants were scanned twice on two separate days. Participants in the control group (N=13) were scanned after drinking a soft-drink at both scanning sessions, while participants in the alcohol group (N=12) were scanned once after drinking an alcoholic beverage resulting in a BAC of 0.02%, and once after drinking an alcoholic beverage resulting in a BAC of 0.08%. A decrease in neuronal activation was seen in the dorsal anterior cingulate cortex (dACC) and in the cerebellum in the alcohol group at the BAC of 0.08% when the participants performed the most demanding task. The dACC is important in cognitive control, working memory, response inhibition, decision making and in error monitoring. The results have revealed that the effect of alcohol intoxication on brain activity is dependent on BAC and of cognitive load.

Aim: To investigate dynamic changes in functional brain activity in mild cognitive impairment (MCI) in response to a graded working memory (WM) challenge with increasing memory load. Methods: In an event-related functional magnetic resonance imaging (fMRI) study, 35 MCI and 22 cognitively normal subjects performed a visuospatial associative WM task with 3 load levels. Potential performance differences were controlled for by individually calibrating the number of items presented at each load. Results: An interaction between group and WM load was observed during stimulus encoding. At lower loads, greater activity in the right anterior cingulate and right precuneus was observed in MCI subjects. As the load increased to higher levels, reduced activation in these regions and greater deactivation in the posterior cingulate-medial precuneus were observed in MCI compared to control subjects. Stronger expression of load-related patterns of activation and deactivation in MCI subjects was associated with greater clinical severity and a more abnormal pattern of performance variability. Conclusion: Patterns of overactivation, underactivation and deactivation during successful encoding in MCI subjects were dependent on WM load. This type of graded cognitive challenge may operate like a ‘memory stress test’ in MCI and may be a useful biomarker of disease at the predementia stage.

Validating and improving the correction of ocular artifacts in electro-encephalography

Mental workload is reflected in driver behaviour, physiology, eye movements and prefrontal cortex activation