- Research Article
65
- 10.1016/j.displa.2008.12.003
Analysing user physiological responses for affective video summarisation
- Dec 24, 2008
- Displays
- Arthur G Money + 1 more +1
Analysing user physiological responses for affective video summarisation
4 physiological measures--electromyogram, respiration rate, heart rate, and skin conductance--were recorded for 11 high and 11 low hypnotizable Ss. It was hypothesized (a) that physiological responsiveness during hypnosis would vary depending on the nature of the task instructions, and (b) that high hypnotizable Ss would show more physiological responsiveness than low hypnotizable Ss. The first hypothesis was substantiated across all 4 measures. Only heart rate levels supported the second hypothesis. The results are discussed as they relate to the 2 hypotheses and to future research.
Analysing user physiological responses for affective video summarisation
Analysing user physiological responses for affective video summarisation
Body and brain responses underlying salient emotional information processing
Emotion is a multifaceted construct comprised of cognitive, physiological and behavioural elements. In the neuroscience of emotion, central (neural) and peripheral (physiological) responses are often studied. The objective characterisation of both these processes separately and concomitantly, is thus a crucial step towards elucidating the neuro-biological underpinnings of emotions.Studying these processes requires induction of an emotional state through appropriate stimuli, classically achieved by presentation of emotionally expressive faces. However, the push for ecological validity in the past few decades has led to development of naturalistic paradigms to induce complex emotional states. These novel stimuli have been employed to capture neuronal and psycho-physiological processes with fMRI, EEG/MEG, PET and heart rate, skin conductance respectively, and which have accordingly yielded fundamental insights into our understanding of the “emotional brain”. However, alternative techniques may add valuable new knowledge whilst also providing further validation and convergence of existing evidence.Employing novel experimental paradigms which emphasize the use of naturalistic stimuli, and using unique data acquisition tools of thermal imaging and intracranial EEG, this thesis addresses the dynamics underlying the neural and peripheral responses underlying emotional states. The thesis comprises seven chapters reporting four experimental studies and a narrative review. In Chapter 1 I first introduce the core research question I address in the context of emotion. In doing so, I introduce the stimuli to evoke emotions in the laboratory setting, the physiological measures to study these and the neuroimaging techniques to probe corresponding brain responses. I then present a brief conceptual description of the analytical methods used in neuroscience and which I have employed subsequently in this thesis.In Chapter 2 I critically review use of naturalistic stimuli in neuroscience. Naturalistic paradigms are an alternative to strict laboratory-controlled stimuli sets of picture and sounds which are strong in simplicity but arguably fall short on ecological validity. Naturalistic paradigms offer greater contextualization of emotions than standard, briefly presented static stimuli. I also describe the methodological developments that the use of naturalistic stimuli has required to build on initial reports. I conclude this chapter with the promise of naturalistic stimuli to provide translational benefits.I follow this with 2 chapters with experimental studies focussed on the physiological responses induced by emotional stimuli. These studies highlight the use of thermal imaging to quantify temperature fluctuations of the faces of participants as a contact-free alternative to the traditional measures of skin conductance and heart rate. Specifically, Chapter 3 provides a validation of a new region of interest (ROI) based facial thermal imaging extraction method based on deep learning algorithms. I validate this method by using an acoustic startle probe showing a decrease in temperature in nose and cheeks replicating previously reported findings. Such ROI based method is hoped to make thermal imaging easier to use for wide application in psychophysiological research.In Chapter 4 I used a naturalistic stimulus to induce dynamic affective states. Here, I quantify physiological correlates (thermal, GSR and heart rate metrics) of the emotional responses. I demonstrate a synchrony in physiological responses of participants and thus by extension their emotional state. I also demonstrate the ability of thermal responses to vary according to emotional states encouraging its uses in emotional state identification. On top of these results, facial thermal responses are quantified by an innovative method of spatial independent component analysis. This method consistently captures a few regions of the face (nose, cheeks, and philtrum) and supersedes ROI based approach as an unbiased data driven method.Moving on from the physiological responses, in Chapter 5 and Chapter 6, I investigate the brain responses underlying emotional processing using a unique and rare neuroimaging technique of stereo-electroencephalography. Performed in clinical contexts, it overcomes the major short comings of EEG and fMRI of poor spatial and temporal resolution respectively when investigating a few brain regions. I use this technique to elucidate the dynamics of the temporal pole (TP) and the amygdala activity underlying emotional processing. Specifically in Chapter 5 I use an affective picture paradigm and in Chapter 6 I use naturalistic stimuli of emotional music and movies. Employing comprehensive hierarchical analyses; probing localised responses, functional connectivity and effective connectivity, I demonstrate that the TP exerts a modulatory influence on the amygdala activity according to valence of emotionally salient stimuli irrespective of the modality of stimuli. The TP function has often been inconsistently reported in literature and these results provide crucial evidence for TP’s vital role in emotional processing.Finally in Chapter 7 I summarise the overall contributions of this thesis, namely; the development of two new novel complementary facial thermal analysis methods; thermal imaging as a contactless tool to capture physiological changes potentially useful in emotional state identification; and the TP’s modulatory influence on amygdala activity according to valence of emotional stimuli emphasizing a vital role of TP in emotion processing via its role as conceptual store.
Read moreImproving Specificity of the Musical Mood Induction Procedure
The musical mood induction procedure was used to induce 3 negative moods: sadness, fatigue and anxiety. Induction was validated using subjective and physiological measures. One hundred twenty-seven participants listened to one of 18 film soundtrack excerpts for 20 minutes. Physiological response (heart rate, respiration, skin conductance level (SCL), and facial electromyography) was recorded throughout the induction and postinduction phases. Subjective mood ratings (sadness, anxiety, tiredness, valence, arousal) were provided before induction and throughout the postinduction phase. Repeated measures ANOVAs showed increase in valence and decrease in arousal in all conditions after induction, which persisted in the postinduction phase, and an increase in tiredness immediately after induction. Reduction in SCL was strongest in the fatigue condition. However, difference between groups was only evident when comparing fatigue and sadness conditions between 3-10 minutes. Lack of between-group differences and mixed physiological findings suggest that specificity is difficult to achieve through musical mood induction.
Read moreHow the presence of a dog and types of interaction affect physiological responses to experimental heat pain induction in healthy humans - a randomized controlled study.
It has become increasingly popular to include dogs as a complement to regular therapy, with the expectation that they offer, among other benefits, pain-relieving effects. Meanwhile, studies covering the topic of painful situations within the field of animal-assisted interventions (AAI) present conflicting results and rarely consider the type and duration of interaction with the dog. Thus, the impact of human-dog interactions on physiological measurements during painful situations is largely unknown. Basic research is needed on the effects of interacting with a dog, using commonly applied immediate physiological measurements in healthy humans during experimental pain induction to fill this gap in the literature. The present study investigated how AAI influences physiological measurements when healthy humans are subjected to experimental heat pain induction. Simultaneously, the study explored how the duration of different types of interaction with a human companion or a dog as well as dog behavior during experimental heat pain induction affected physiological measurements. Fifty-eight healthy participants (14 men, 44 women, age: 18-66 years) were randomly assigned to one of two intervention groups: 1) a dog and a human companion or 2) a human companion only. Both intervention groups underwent two test conditions in a balanced order: an active test condition with their allocated intervention and a control test condition without their allocated intervention. The participants were exposed to a 5-minute heat pain induction trial in both test conditions with a 20-minute break between trials. Heart rate (HR), heart rate variability (HRV) and skin conductance (SC), analyzed as tonic level (SCL) and peak counts (SCR), were continuously recorded. Blood pressure (BP) and salivary cortisol (s-cortisol) were collected as pre- and post-measurements for each test condition. Behavioral interactions between the participant, dog and human companion as well as behavior of the dog were recorded and the influence of the behavioral interactions on each physiological measure was analyzed. Linear Mixed Models were applied. HR was higher for the intervention group with a dog and a human companion compared to a human companion only (p=0.013). Additionally, within-subject comparison showed an increased HR during the active condition compared to the control condition in both intervention groups (dog and human companion: p<0.001 and human companion only: p=0.025). None of the other physiological measurements were influenced by the mere presence of a dog in either the between- nor within-subject comparisons. Within the human companion only group, SCL and SCR were higher during the active condition compared to the control (both p<0.001). The duration of the behavioral interaction between the participant and human companion variously influenced HRV, SBP, s-cortisol and SCL during the active condition in both intervention groups. Further, the duration of the behavioral interactions with the dog and dog behavior variously influenced HR, HRV, s-cortisol and SCL during the active condition in the human companion and the dog intervention group. In conclusion, this study shows that the presence of a dog in addition to a human companion during experimental heat pain induction results in an increased HR compared to the presence of a human companion only. Thus, despite previously reported pain-relieving effects of the presence of a dog, this study suggests that dog presence may also induce a certain level of arousal and further studies are needed to explore the causal mechanisms. Interactions with a dog or a human companion influenced several physiological measurements and it is therefore important to quantify the type and duration of human-animal interaction in AAI-studies.
Read moreIdentifying Relationships between Physiological Measures and Evaluation Metrics for 3D Interaction Techniques
This project aims to present a methodology to study the relationships between physiological measures and evaluation metrics for 3D interaction techniques using methods for multivariate data analysis. Physiological responses, such as heart rate and skin conductance, offer objective data about the user stress during interaction. This could be useful, for instance, to evaluate qualitative aspects of interaction techniques without relying on solely subjective data. Moreover, these data could contribute to improve task performance analysis by measuring different responses to 3D interaction techniques. With this in mind, we propose a methodology that defines a testing protocol, a normalization procedure and statistical techniques, considering the use of physiological measures during the evaluation process. A case study comparison between two 3D interaction techniques (ray-casting and HOMER) shows promising results, pointing to heart rate variability, as measured by the NN50 parameter, as a potential index of task performance. Further studies are needed in order to establish guidelines for evaluation processes based on welldefined associations between human behaviors and human actions realized in 3D user interfaces.
Read moreAmbulatory physiological measures obtained under naturalistic urban mobility conditions have acceptable reliability.
Ambulatory assessment methods in psychology and clinical neuroscience are powerful research tools for collecting data outside of the laboratory. These methods encompass physiological, behavioral, and self-report measures obtained while individuals navigate in real-world environments, thereby increasing the ecological validity of experimental approaches. Despite the recent increase in applications of ambulatory physiology, data on the reliability of these measures is still limited. To address this issue, twenty-six healthy participants (N = 15 female, 18-34 years) completed an urban walking route (distance M = 2.2km, ±SD = 0.11; duration M = 30.8min, ±SD = 1.34; temperature M = 18.34° degree Celsius, ±SD = 1.19, Range = 16°-21° degrees Celsius) on two separate testing days, while assessing the effect of metabolic state (sated vs. fasted). GPS-location and ambulatory physiological measures (cardiovascular and electrodermal activity) were continuously recorded. The results showed no significant differences in single physiological measures between fasted and sated states. Bootstrapped test-retest reliabilities of single measures and aggregate scores derived via principal component analysis (PCA) were computed. The first principal component (PC#1) accounted for 39-45% of variance across measures. PC#1 scores demonstrated an acceptable test-retest reliability (r = .60) across testing days, exceeding the reliabilities of most individual measures (heart rate: r = .53, heart rate variability: r = .50, skin conductance level: r = .53, no. of skin conductance responses: r = .28, skin conductance response amplitude: r = .60). Results confirm that ambulatory physiological measures recorded during naturalistic navigation in urban environments exhibit acceptable test-retest reliability, in particular when compound scores across physiological measures are analyzed, a prerequisite for applications in (clinical) psychology and digital health.
Read moreResearch on electrodermal activity
In this chapter, a method of physiological measurements—that is detection of electrodermal activity based on the sectonic activity of eccrine sweat glands—is discussed. It is believed that the excretion of sweat, which is regulated by the nervous system acting independently of human will, is an indicator of a person’s emotional arousal as a result of specific stimuli. Hence, the electrodermal reaction can be used in diagnosing emotional arousal caused by, e.g. specific products, advertisements or elements of the in-store space. Electrical activity of the skin is caused by two types of stimuli: sustained and one-off. Sustained stimuli have a continuous effect on the body over a relatively long period of time. On the other hand, one-off stimuli have a relatively strong and very short-lasting effect. This type is defined as novel, unexpected, significant or aversive. Electrodermal activity is measured on the skin surface (Strelau, 2006). Generally speaking, the measurement of electrodermal activity is one of the biometric measurements. Biometrics is a universal term that represents measurements of the body’s physiological responses—not directly of the brain—to external stimuli that are felt through the senses (Pradeep, 2010; Berčík & Rybanská, 2017). The electrodermal method allows to measure either electrical resistance or its inverse, i.e. the electrical conductivity of the skin. These measurements are carried out while a small current flows through the skin from an external source. Electrodermal activity measurement is performed with the use of special electrodes, electrode gels and recording devices. The available equipment for the analysis of electordermal activity is characterised by relatively low cost (compared to other devices for physiological measurements)of purchase and operation. Moreover, the electrodermal activity measurement is non-invasive and carries no risk to the health or life of the test subjects.
Read moreEffect of a Virtual Biophilic Residential Environment on the Perception and Responses of Seniors
This study investigates the effects of a virtual biophilic residential environment on seniors’ physiological and subjective responses to evaluate its potential to promote healing and recovery. Thirty seniors were exposed to three different scales (units, buildings, complexes) of virtual biophilic residential environments that combined both physical and digital biophilic elements. Physiological responses, including heart rate, heart rate variability, and galvanic skin response, were measured alongside self-reported levels of satisfaction and immersion. The primary objective was to assess the effectiveness of physical and digital design interventions at each residential scale. The findings revealed that the virtual biophilic residential environment reduced physiological stress in seniors, with the most significant impact observed at the unit scale. Digital design interventions further enhance stress relief benefits, indicating that integrating physical and digital elements in biophilic residential environments can positively influence seniors’ stress levels. Additionally, significant correlations were identified between physiological responses and subjective perceptions of immersion and satisfaction. This study is valuable as an initial comparative analysis of the effectiveness of physical and digital approaches in biophilic design. This paper is a preliminary study and is significant in that it systematizes virtual environment research from an age-friendly perspective and expands approaches to biophilic design.
Read moreVehicle Control and Driving Experience a Psychophysiological Approach
An instrumented vehicle was used for real-time recording of drivers physiological characteristics (galvanic skin response, heart rate and muscle activity), steering and braking behavior and vehicle response (speed, distance travelled and triaxial accelerations). Thirtythree drivers performed test-drives along a 25 km stretch of rural road. Seventeen of the drivers were inexperienced and the rest experienced. Response values of vehicle control variables were similar in both groups, whereas physiological responses displayed large differences. Using multiple regression techniques it is shown that for the inexperienced group, road-traffic situations involving the use of the brake explain most of the variance in the physiological measures. Similarly, the steering response was the most important variable for the experienced group. GSR seems to be an efficient indicator of the mental effort involved in driving. Control skills develop rapidly whereas the discrepancies in physiological responses indicate the relatively slow development of skills necessary for collecting the relevant information.
Read moreEffects of hearing acuity on psychophysiological responses to effortful speech perception
In recent studies, psychophysiological measures have been used as markers of listening effort, but there is limited research on the effect of hearing loss on such measures. The aim of the current study was to investigate the effect of hearing acuity on physiological responses and subjective measures acquired during different levels of listening demand, and to investigate the relationship between these measures. A total of 125 participants (37 males and 88 females, age range 37–72 years, pure-tone average hearing thresholds at the best ear between -5.0 to 68.8 dB HL and asymmetry between ears between 0.0 and 87.5 dB) completed a listening task. A speech reception threshold (SRT) test was used with target sentences spoken by a female voice masked by male speech. Listening demand was manipulated using three levels of intelligibility: 20 % correct speech recognition, 50 %, and 80 % (IL20 %/IL50 %/IL80 %, respectively). During the task, peak pupil dilation (PPD), heart rate (HR), pre-ejection period (PEP), respiratory sinus arrhythmia (RSA), and skin conductance level (SCL) were measured. For each condition, subjective ratings of effort, performance, difficulty, and tendency to give up were also collected. Linear mixed effects models tested the effect of intelligibility level, hearing acuity, hearing asymmetry, and tinnitus complaints on the physiological reactivity (compared to baseline) and subjective measures. PPD and PEP reactivity showed a non-monotonic relationship with intelligibility level, but no such effects were found for HR, RSA, or SCL reactivity. Participants with worse hearing acuity had lower PPD at all intelligibility levels and showed lower PEP baseline levels. Additionally, PPD and SCL reactivity were lower for participants who reported suffering from tinnitus complaints. For IL80 %, but not IL50 % or IL20 %, participants with worse hearing acuity rated their listening effort to be relatively high compared to participants with better hearing. The reactivity of the different physiological measures were not or only weakly correlated with each other. Together, the results suggest that hearing acuity may be associated with altered sympathetic nervous system (re)activity. Research using psychophysiological measures as markers of listening effort to study the effect of hearing acuity on such measures are best served by the use of the PPD and PEP.
Read morePhysiological Response of Cattle to Climatic Stress
Physiological responses are the immediate response shown by livestock against climatic stress and categorized as rectal temperature, respiration rate, heart rate, sweating rate, skin temperature along with digestion and absorption of nutrients. Rectal temperature is regarded as sensitive indicator of physiological status during climatic stress in domestic species. Respiration rate is considered as most consistent biomarker amongst the different physiological responses shown by livestock’s against climatic stress. An increase in respiration rate was noticed in cattle during heat stress. Variation in heart rate and pulse rate are appraised as indicator in livestock’s against climatic stress. Various environmental variables modulate sweating rate such as ambient temperature, relative humidity, wind velocity and solar radiation. Generally, the skin temperature was increases with increase in ambient temperature. The chapter describes the physiological and endocrine responses in which helps in acquisition of thermo-tolerance to livestock’s.
Read moreAlprazolam reduces response to loud tones in panic disorder but not in posttraumatic stress disorder
Alprazolam reduces response to loud tones in panic disorder but not in posttraumatic stress disorder
Development of a novel startle response task in Duchenne muscular dystrophy.
Duchenne muscular dystrophy (DMD), an X-linked childhood-onset muscular dystrophy caused by loss of the protein dystrophin, can be associated with neurodevelopmental, emotional and behavioural problems. A DMD mouse model also displays a neuropsychiatric phenotype, including increased startle responses to threat which normalise when dystrophin is restored in the brain. We hypothesised that startle responses may also be increased in humans with DMD, which would have potential translational therapeutic implications. To investigate this, we first designed a novel discrimination fear-conditioning task and tested it in six healthy volunteers, followed by male DMD (n = 11) and Control (n = 9) participants aged 7-12 years. The aims of this methodological task development study were to: i) confirm the task efficacy; ii) optimise data processing procedures; iii) determine the most appropriate outcome measures. In the task, two neutral visual stimuli were presented: one 'safe' cue presented alone; one 'threat' cue paired with a threat stimulus (aversive noise) to enable conditioning of physiological startle responses (skin conductance response, SCR, and heart rate). Outcomes were the unconditioned physiological startle responses to the initial threat, and retention of conditioned responses in the absence of the threat stimulus. We present the protocol development and optimisation of data processing methods based on empirical data. We found that the task was effective in producing significantly higher physiological startle SCR in reinforced 'threat' trials compared to 'safe' trials (P < .001). Different data extraction methods were compared and optimised, and the optimal sampling window was derived empirically. SCR amplitude was the most effective physiological outcome measure when compared to SCR area and change in heart rate, with the best profile on data processing, the least variance, successful conditioned response retention (P = .01) and reliability assessment in test-retest analysis (rho = .86). The definition of this novel outcome will allow us to study this response in a DMD population.
Read moreResearch on the effect of biosensing technology on the dissemination of health information in ideological and political education
Biosensing technologies, which monitor physiological responses such as Heart Rate Variability (HRL), Skin Conductance Level (SCL), and Electroencephalogram (EEG) activity, offer a novel approach to enhancing the dissemination of health information in ideological and political education (IPE). In this context, health information encompasses topics such as mental health, stress management, and healthy lifestyle practices, all crucial to students’ overall well-being. Traditional health education methods cannot often capture real-time physiological and emotional responses, which can improve engagement and learning outcomes. This research explores the effectiveness of biosensing technology in enhancing the dissemination of health information within IPE. It examines how physiological data can be utilized to assess student engagement, emotional responses, and learning outcomes related to health. A mixed-methods approach was adopted, combining quantitative data from wearable biosensors (heart rate monitors, Galvanic Skin Response (GSR) sensors, EEG headsets) with qualitative feedback from students. Physiological data were preprocessed using signal filtering techniques, such as the Savitzky-Golay Filter, and features such as heart rate variability, skin conductance, and EEG alpha waves were extracted using the Kalman Filter (KF). A Modified Runge-Kutta Optimizer Integrated with Deep Belief Networks (MRKO-DBN) classifier was employed to predict student engagement based on these features. The research revealed that physiological responses, particularly heart rate variability and skin conductance, were strongly correlated with student engagement. The MRKO-DBN model achieved accuracy in predicting engagement. Qualitative feedback further confirmed that Biosensing technology significantly improved students’ engagement. Integrating Biosensing technology into health education within ideological and political contexts offers significant potential for enhancing student engagement and learning outcomes. By providing real-time, personalized feedback, it fosters a more interactive and responsive learning environment.
Read moreExplicit and Implicit Responses to Tasting Drinks Associated with Different Tasting Experiences
Probing food experience or liking through verbal ratings has its shortcomings. We compare explicit ratings to a range of (neuro)physiological and behavioral measures with respect to their performance in distinguishing drinks associated with different emotional experience. Seventy participants tasted and rated the valence and arousal of eight regular drinks and a “ground truth” high-arousal, low-valence vinegar solution. The discriminative power for distinguishing between the vinegar solution and the regular drinks was highest for sip size, followed by valence ratings, arousal ratings, heart rate, skin conductance level, facial expression of “disgust,” pupil diameter, and Electroencephalogram (EEG) frontal alpha asymmetry. Within the regular drinks, a positive correlation was found between rated arousal and heart rate, and a negative correlation between rated arousal and Heart Rate Variability (HRV). Most physiological measures showed consistent temporal patterns over time following the announcement of the drink and taking a sip. This was consistent over all nine drinks, but the peaks were substantially higher for the vinegar solution than for the regular drinks, likely caused by emotion. Our results indicate that implicit variables have the potential to differentiate between drinks associated with different emotional experiences. In addition, this study gives us insight into the physiological temporal response patterns associated with taking a sip.
Read more