Evaluation of physiological response and synchronisation errors during synchronous and pseudosynchronous stimulation trials

Continuous daily stress and high workload can have negative effects on individuals' physical and mental well-being. It has been shown that physiological signals may support the prediction of stress and workload. However, previous research is limited by the low diversity of signals concurring to such predictive tasks and controlled experimental design. In this paper we present 1) a pipeline for continuous and real-life acquisition of physiological and inertial signals 2) a mobile agent application for on-the-go event annotation and 3) an end-to-end signal processing and classification system for stress and workload from diverse signal streams. We study physiological signals such as Galvanic Skin Response (GSR), Skin Temperature (ST), Inter Beat Interval (IBI) and Blood Volume Pulse (BVP) collected using a non-invasive wearable device; and inertial signals collected from accelerometer and gyroscope sensors. We combine them with subjects' inputs (e.g. event tagging) acquired using the agent application, and their emotion regulation scores. In our experiments we explore signal combination and selection techniques for stress and workload prediction from subjects whose signals have been recorded continuously during their daily life. The end-to-end classification system is described for feature extraction, signal artifact removal, and classification. We show that a combination of physiological, inertial and user event signals provides accurate prediction of stress for real-life users and signals.

Analysing user physiological responses for affective video summarisation

The prevalence of mental fatigue is a noteworthy phenomenon that can affect individuals across diverse professions and working routines. This paper provides a comprehensive dataset of physiological signals obtained from 23 participants during their professional work and questionnaires to analyze mental fatigue. The questionnaires included demographic information and Chalder Fatigue Scale scores indicating mental and physical fatigue. Both physiological signal measurements and the Chalder Fatigue Scale were performed in two sessions, morning and evening. The present dataset encompasses diverse physiological signals, including electroencephalogram (EEG), blood volume pulse (BVP), electrodermal activity (EDA), heart rate (HR), skin temperature (TEMP), and 3-axis accelerometer (ACC) data. The NeuroSky MindWave EEG device was used for brain signals, and the Empatica E4 smart wristband was used for other signals. Measurements were carried out on individuals from four different occupational groups, such as academicians, technicians, computer engineers, and kitchen workers. The provision of comprehensive metadata supplements the dataset, thereby promoting inquiries about the neurophysiological concomitants of mental fatigue, autonomic activity patterns, and the repercussions of a cognitive burden on human proficiency in actual workplace settings. The accessibility of the aforementioned dataset serves to facilitate progress in the field of mental fatigue research while also laying the groundwork for the creation of customized fatigue evaluation techniques and interventions in diverse professional domains.

Chronic pain is prevalent and disproportionately impacts adults with a lower quality of life. Although subjective self-reporting is the "gold standard" for pain assessment, tools are needed to objectively monitor and account for inter-individual differences. This study introduced a novel framework to objectively classify pain intensity levels using physiological signals during Quantitative Sensory Testing sessions. Twenty-four participants participated in the study wearing physiological sensors (blood volume pulse (BVP), galvanic skin response (GSR), electromyography (EMG), respiration rate (RR), skin temperature (ST), and pupillometry). This study employed two analysis plans. Plan 1 utilized a grid search methodology with a 10-fold cross-validation framework to optimize time windows (1-5 s) and machine learning hyperparameters for pain classification tasks. The optimal time windows were identified as 3 s for the pressure session, 2 s for the pinprick session, and 1 s for the cuff session. Analysis Plan 2 implemented a leave-one-out design to evaluate the individual contribution of each sensor modality. By systematically excluding one sensor's features at a time, the performance of these sensor sets was compared to the full model using Wilcoxon signed-rank tests. BVP emerged as a critical sensor, significantly influencing performance in both pinprick and cuff sessions. Conversely, GSR, RR, and pupillometry demonstrated stimulus-specific sensitivity, significantly contributing to the cuff session but with limited influence in other sessions. EMG and ST showed minimal impact across all sessions, suggesting they are non-critical and suitable for reducing sensor redundancy. These findings advance the design of sensor configurations for personalized pain management. Future research will focus on refining sensor integration and addressing stimulus-specific physiological responses.

The article discusses various aspects and provides a theoretical analysis of research on the development of musical abilities and their impact on personality psychology. Several studies point to potential mechanisms underlying the development of musical skills and the influence of music on brain activity. The interest of scientists in the problem of developing musical abilities in personality is highlighted within the context of neuroscience, the influence of the emotional sphere on music perception, and the formation of musical preferences in both children and adults. The article examines the issue of musical ability development and its age-specific characteristics, their structure, and the role of general abilities in musical development. The article aims to examine and systematize the theoretical aspects related to the development of musical abilities in individuals, to study contemporary theories, to consolidate scientific approaches, and to deepen research in this field. The research provides definitions of the concept of "musical abilities" by representatives of various fields of knowledge: psychology, musicology, music pedagogy, as well as their classification. A characterization of musicality as a special mental function and the concept of the synesthetic nature of musicality are provided. It has been proven that musicality influences personality development, making it more integrated and balanced, fostering the development of various aspects: intellectual, emotional, social, and creative. It is noted that a person's musical giftedness is based on sensory musical abilities, the main ones being musical ear, sense of rhythm, and musical memory. The article examines the issue of developing musical ear - the most important sensory musical ability, the development of which is associated with a high level of human mental evolution. A number of scientific studies on the perception of musical rhythm are analyzed, and the development of a sense of musical rhythm in children is discussed. The definition of the concept of musical memory as a separate musical ability is provided, describing its classification, functioning, and interrelated types. Emphasis is placed on the necessity of continual training for its development and improvement. The concept of musical thinking and musical imagination is separately examined, reflecting aspects of cognitive and creative activity in music. Conclusions are drawn regarding the development of musical abilities as a universal tool for personal growth.

This study proposes a methodology in building a multimodal stress-level model using different non-invasive physiological signals: Galvanic Skin Response (GSR), Blood Volume Pulse (BVP), and Respiratory Variability (Resp). Paced Stroop, Mental Math, and Game were used to induce stress to 4 subjects. A fixed window size of 5 seconds and 1 second sliding size were used to segment the self-annotated data. Six statistical features were extracted from each physiological signals, giving a total of 18 features. Different classification and regression algorithms were used to get the correlation of the features and the stress labels. The results show very high correlation for all three stress inducers with Paced Stroop Test as the highest which in term means that all 3 activities are effective in inducing stress. Results also show a good correlation for the 3 physiological signals with Respiration as the highest among the 2 others, which indicate that Respiration provides better correlation even by just using low-level features.

This paper explores a novel method for anxiety detection in older adults using simple wristband sensors such as electrodermal activity (EDA) and photoplethysmogram (PPG) and a context-based feature. The proposed method for anxiety detection combines features from a single physiological signal with an experimental context-based feature to improve the performance of the anxiety detection model. The experimental data for this work are obtained from a year-long experiment on 41 healthy older adults (26 females and 15 males) in the age range 60–80 with mean age \(73.36 \pm 5.25\) during a Trier Social Stress Test (TSST) protocol. The anxiety level ground truth was obtained from State–Trait Anxiety Inventory (STAI), which is regarded as the gold standard to measure perceived anxiety. EDA and blood volume pulse (BVP) signals were recorded using a wrist-worn EDA and PPG sensor, respectively. 47 features were computed from EDA and BVP signal, out of which a final set of 24 significantly correlated features were selected for analysis. The phases of the experimental study are encoded as unique integers to generate the context feature vector. A combination of features from a single sensor with the context feature vector is used for training a machine learning model to distinguish between anxious and not-anxious states. Results and analysis showed that the EDA and BVP machine learning models that combined the context feature along with the physiological features achieved 3.37% and 6.41% higher accuracy, respectively, than the models that used only physiological features. Further, end-to-end processing of EDA and BVP signals was simulated for real-time anxiety level detection. This work demonstrates the practicality of the proposed anxiety detection method in facilitating long-term monitoring of anxiety in older adults using low-cost consumer devices.

This study addresses the question whether galvanic skin response (GSR) and blood volume pulse (BVP) of untrained and unaided observers can be used to identify real and posed smiles from different sets of smile videos or smile images. Observers were shown smile face videos/images, either singly or paired, with the intention to recognise each viewed as real or posed smiles. We created four experimental situations, namely single images (SI), single videos (SV), paired images (PI), and paired videos (PV). The GSR and BVP signals were recorded and processed. Our machine learning classifiers reached the highest accuracy of 93.3%, 87.6%, 92.0%, 91.7% for PV, PI, SV, and SI respectively. Finally, PV and SI were found to be the easiest and hardest way to identify real and posed smiles respectively. Overall, we demonstrated that observers’ subconscious physiological signals (GSR and BVP) are able to identify real and posed smiles at a good accuracy.KeywordsPhysiological signalsAffective computingClassificationMachine learningSmile

Despite the advantages of using physiological sensors to collect emotion data, emotion recognition systems using physiological signals such as Electrodermal Activity (EDA), Electrocardiogram (ECG) or Electromyography (EMG) are mainly tested in controlled environments or under laboratory conditions. The use of physiological data in real-world scenarios has not been widely investigated. One of the main issues of using physiological data from real-world scenarios is that the data may also be influenced by movement and in some cases, the physiological response to emotions can be even confused with the one due to physical activities, such as walking or running. In this paper, we investigate the impact of physical activities in the recognition of emotions and provide new insights on how emotion data from physiological sensors are affected by these activities. We use two scenarios (one with and one without the influence of physical movement) to investigate the effect of physical activities in the Blood Volume Pulse (BVP) and the Skin Temperature (TMP) signals. To overcome these issues we used a random forest algorithm to model both scenarios. Our results show that by combining emotion data from both scenarios, we can achieve a recognition accuracy of up to 96%.

This dataset provides demographic (age and sex), psychometric, and physiological data from 28 participants (20 males, 8 females) aged 19–27 years. Psychometric assessments include four questionnaires: the Emotion Regulation Questionnaire, Self-Assessment Manikin, and two perception-related scales (EmFacs and Uncanny Valley). Physiological signals recorded include electroencephalography, blood volume pulse, galvanic skin response, and skin temperature. Emotion induction was conducted using two virtual environments designed to elicit distinct emotional states: happiness (an outdoor natural forest) and anger (a subway car), with emotional stimuli delivered through speech by a digital human. Physiological responses were recorded at baseline and during immersion in the virtual environments. This dataset can support research on affective computing, human-computer interaction, and emotion recognition.

Recent research has shown that design principles inherent in a Japanese style garden can reduce measures of stress in both healthy and dementia patients. However, it was not clear how subjects' visual interaction with the scene affected their physiological responses. To address that, we developed a novel non-invasive system to collect synchronized measurements of eye gaze and physiological indicators of sympathetic neural activity: the electrocardiogram, the blood volume pulse and the galvanic skin response, as subjects view a garden environment. We characterized the visual engagement of subjects using the average fixation duration, the saccade amplitude and the gaze transition entropy. We find a statistically significant positive correlation between gaze transition entropy and mean heart rate change. Our results suggest that the visual engagement of subjects with their environment may influence their physiological responses to it: more engagement may lead to more relaxation. Our results also highlight the importance of taking into account the detailed spatio-temporal characteristics of the gaze trajectory.

BackgroundCoronary angiography is an important tool in diagnosis of cardiovascular diseases. However, it is the administration is relatively stressful and emotionally traumatic for the subjects. The aim of this study is to evaluate psychophysiological responses induced by the coronary angiography instead of subjective methods such as a questionnaire. We have also evaluated the influence of the tranquilizer on the psychophysiological responses.MethodsElectrocardiography (ECG), Blood Volume Pulse (BVP), and Galvanic Skin Response (GSR) of 34 patients who underwent coronary angiography operation were recorded. Recordings were done at three phases: "1 hour before," "during," and "1 hour after" the coronary angiography test. Total of 5 features obtained from the physiological signals were compared across these three phases. Sixteen of the patients were administered 5 mg of a tranquilizer (Diazepam) before the operation and remaining 18 were not.ResultsOur results indicate that there is a strong correlation between features (LF/HF, Bk, DN1/DN2, skin conductance level and seg_mean) in terms of reflecting psychophysiological responses. However only DN1/DN2 feature has statistically significant differences between angiography phases (for diazepam: p = 0.0201, for non_diazepam p = 0.0224). We also note that there are statistically significant differences between the diazepam and non-diazepam groups for seg_mean features in "before", "during" and "after" phases (p = 0.0156, 0.0282, and 0.0443, respectively).ConclusionsThe most intense sympathetic activity is observed in the "during" angiography phase for both of the groups. The obtained features can be used in some clinical studies where generation of the customized/individual diagnoses styles and quantitative evaluation of psychophysiological responses is necessary.

Machine learning based acute stress detection systems use physiological sensor data to objectively predict acute stress. However, machine learning algorithms developed for stress detection do not consider how machine learning algorithm performance may be affected based on a change(s) in the deployment environment. In this study, the deployment environment changes that are investigated are sensor type and sensor placement. Electrodermal activity (EDA) and skin temperature (TEMP) data from two different sensors, the RespiBAN Professional (RespiBAN) and the Empatica E4 are used to train three different machine learning models. The RespiBAN records the EDA data from the rectus abdominis and records the skin TEMP data from the sternum. The Empatica E4 sensor records both EDA and skin TEMP data from the wrist. Three different support vector machine (SVM) models were trained to classify no-stress versus stress states using EDA and skin TEMP data. The first model was trained using data from the RespiBAN wearable sensor (SVM-R), the second model was trained using data from the Empatica E4 sensor (SVM-E) and third model was trained using data from both sensors (SVM-RE). The accuracy of SVM-R on a test set recorded by the RespiBAN sensor was 100%. The accuracy of SVM-E on a test set recorded by the Empatica E4 sensor was 99%. The accuracy of SVM-RE on a test set recorded by both the RespiBAN and Empatica E4 sensor was 82%. The accuracy of the SVM-R on a test set recorded by the Empatica E4 was 64%. These results suggest that research and development cannot be hardware or placement agnostic with wearable sensing data. Sensor type and placement must be taken into consideration when reporting performance metrics of physiological based stress detection machine learning algorithms.

Enhancing user experience and performance, including task load in immersive environments, requires accurate prediction of user gaze point, reaction time, and mental and physical load uptake. Current gaze prediction approaches focus primarily on motion-based information, lacking physiological data, which leads to poor prediction accuracy in highly dynamic virtual reality (VR) environments. Traditional cognitive load measurements rely on post-task analysis without proper multimodal data integration and fail to capture the real-time dynamics of user states during interaction. Likewise, reaction time or attention load are often assessed only after the interaction, without using real-time immersive sensor data, which limits adaptive responsiveness. To tackle these limitations, we leveraged a comprehensive multimodal dataset - VRWalking, which recorded timestamped eye-tracking metrics, physiological signals (heart rate and galvanic skin response), and behavioral performance data during real-time engagement in a VR environment. We developed a unified multitask model based on the MultiPatchFormer architecture, which processes multimodal VR signals through dual patch projection branches for gaze and classification inputs. The model employs multiscale patch embeddings, cross-attention between gaze and classification pathways, channel attention, and transformer encoders to jointly predict continuous user gaze and classify reaction time, cognitive load (mental load and physical load). Our methodology achieved excellent predictive performance: 95.64% for reaction time, 98.01% for mental load, and 97.45% for physical load, with a MAPE (Mean Absolute Percentage Error) of 15.24% for gaze prediction. We applied Shapley Additive explanations (SHAP) analysis to interpret the model’s behavior across all features, including eye-tracking, head-tracking, and physiological signals. The analysis revealed which features most influenced the predictions of user gaze, reaction time, mental load, and physical load. Our methods, while based only on the VRWalking dataset, demonstrated strong performance across all tasks, suggesting promising potential for real-world VR applications such as interactive training systems that respond to user attention lapses, educational platforms that adapt to cognitive load, and performance assessments that consider physiological indicators.

BackgroundIn our Dynamic Pathways, account, we hypothesized that childhood stuttering reflects an impairment in speech sensorimotor control that is conditioned by cognitive, linguistic, and emotional factors. The purpose of this study was to investigate potential differences in levels of sympathetic arousal during performance of speech and non-speech tasks between children who do and do not stutter.MethodsSeventy-two preschool-aged children participated in the study, 47 children who stutter (CWS; 38 boys) and 25 children who do not stutter (CWNS; 18 boys). We recorded skin conductance and blood pulse volume (BPV) signals, indices of sympathetic arousal, during higher/lower load speech tasks (structured sentence production and picture description) and non-speech tasks (jaw wagging and forceful blowing). We included a measure that reflects children’s attitudes about their communication skills and a parent-report assessment of temperament.ResultsWe found no significant differences between preschool CWS and CWNS in phasic skin conductance response amplitude or frequency, BPV, and pulse rate for any of the experimental tasks. However, compared to CWNS, CWS had, on average, significantly higher skin conductance levels (SCL), indexing slowly changing tonic sympathetic activity, across both speech and non-speech experimental conditions. We found distinctive task-related profiles of sympathetic arousal in both groups of preschool children. Most children produced the highest levels of sympathetic arousal in the physically demanding blowing task rather than in speech, as seen in previous studies of adults. We did not find differences in temperament between the two groups of preschool children nor a relationship among behavioral indices of temperament and communication attitude and physiological measures of sympathetic arousal.ConclusionWe did not find that atypically high, speech-related sympathetic arousal is a significant factor in early childhood stuttering. Rather, CWS had higher, on average, task-related tonic SCLs across speech and non-speech tasks. A relationship among behavioral measures of temperament and physiological measures of sympathetic arousal was not confirmed. Key questions for future experiments are how the typical coupling of sympathetic and speech sensorimotor systems develops over childhood and adolescence and whether task related developmental profiles follow a different course in children who continue to stutter.