Empathy is increasingly recognized as a key factor in human–AI communication, yet conventional approaches to “digital empathy” often focus on simulating internal, human like emotional states while overlooking the inherently subjective, contextual, and relational facets of empathy as perceived by users. In this work, we propose a human-centered taxonomy that emphasizes observable empathic behaviors and introduce a new dataset, SENSE-7, of real-world conversations between information workers and Large Language Models (LLMs), which includes per-turn empathy annotations directly from the users, along with user characteristics, and contextual details, offering a more user-grounded representation of empathy. Analysis of 695 conversations from 109 participants reveals that empathy judgments are highly individualized, context-sensitive, and vulnerable to disruption when conversational continuity fails or user expectations go unmet. To promote further research, we provide a subset of 672 anonymized conversation and provide exploratory classification analysis, showing that an LLM-based classifier can recognize 5 levels of empathy with an encouraging average Spearman ρ = 0.369 and Accuracy = 0.487 over this set. Overall, our findings underscore the need for AI designs that dynamically tailor empathic behaviors to user contexts and goals, offering a roadmap for future research and practical development of socially attuned, human-centered artificial agents.

Depression is a common and serious mental disorder, and the ocular image is often utilized in early intervention and adjunctive treatment for depression due to its direct association with cognitive processes. Despite recent advancements in ocular movement detection for depression, exploration of enhancing depression detection through simulated visual cognitive processing using ocular movement patterns induced by emotional stimuli remains limited. Deep learning algorithms can enable non-contact and cost-effective early detection of depression by extracting subtle changes in ocular images. In this work, we introduce a novel stimulus-guided depression detection method (SguiNet) that enhances the mining of specific ocular movement patterns through visual processing modalities. The framework employs semantic pseudo-labels of emotional stimulus images to assist in accurately capturing the affective polarity exhibited by depressed and non-depressed participants during emotional stimulation, thereby enabling effective depression detection. First, an Adaptive Temporal Pyramid Module (ATPM) is proposed for extracting high-level spatio-temporal features from ocular video data. Subsequently, a Time Scale Cross Attention Module (TSCAM) is employed to enhance the feature extraction capability of the model. Finally, the model is jointly trained with depression classification based on the generated semantic pseudo-labels and real labels by the Stimulus Paradigm Supervision Module (SPSM). We utilize data collected from a study on computer-aided diagnosis, where participants' ocular movements were recorded while they viewed stimulus images. The results demonstrate that the proposed SguiNet method effectively models ocular movement features with discriminative power for depression detection and provides new insights into ocular movement pattern-assisted depression detection. On a dataset collected from three medical institutions, SguiNet achieves an accuracy of 82.99%, a precision of 80.74%, and an F1-score of 83.85%, highlighting its superior detection capability.

Electroencephalography (EEG) has been established as the primary modality in the field of affective computing. Integrating EEG with electrocardiogram (ECG) signals based on heart-brain coupling can overcome the inherent limitations of unimodal approaches by leveraging complementary neural dynamics, achieving more robust emotion representation. However, the existing methods face two critical challenges. First, poor modeling of psychophysiological spatial correlations limits high-order cognitive feature extraction. Second, the existing methods struggle to capture deep cross-modal interactions due to inherent modality heterogeneity. To overcome these two challenges, this study proposes a multi-view cognitive prior graph convolutional network (MCP-GCN), which adopts domain generalization for emotion recognition. Particularly, two branch graphs are constructed: a functional connectivity branch based on neuropsychological knowledge and a data-driven branch with dynamic feature enhancement. The data-driven branch graph addresses the problem of modality heterogeneity using three constraint mechanisms. The MCP-GCN employs graph convolutional networks with multi-readout functions to capture local and global cognitive state features simultaneously. An attention-based fusion mechanism is developed to combine graph representations from both branches, which allows for enhancing affectiveembed dings. In addition, domain generalization methods are designed that explicitly consider subject-related covariates to extract subject-invariant emotional representations. The cross-subject evaluations achieve the accuracy of 96.00% (DREAMER) and 96.23% (MAHNOB-HCI), with subject-independent tests' accuracy reaching 98.03% (DREAMER) and 98.27% (MAHNOB HCI), demonstrating the state-of-the-art performance across both datasets. Finally, visualization results of multimodal connectivity matrices and graph structures reveal emotion-sensitive heart brain coupling, supporting biological interpretability of the MCP GCN.

Existing manual labeling of micro-expressions is subject to errors in accuracy, especially in cross-cultural scenarios where deviation in labeling of key frames is more prominent. To address this issue, this paper presents a novel Global Anti-Monotonic Differential Selection Strategy (GAMDSS) architecture for enhancing the effectiveness of spatio-temporal modeling of micro-expressions through keyframe re-selection. Specifically, the method identifies Onset and Apex frames, which are characterized by significant micro-expression variation, from complete micro-expression action sequences via a dynamic frame reselection mechanism. It then uses these to determine Offset frames and construct a rich spatio-temporal dynamic representation. A two-branch structure with shared parameters is then used to efficiently extract spatio-temporal features. Extensive experiments are conducted on seven widely recognized micro-expression datasets. The results demonstrate that GAMDSS effectively reduces subjective errors caused by human factors in multicultural datasets such as SAMM and 4DME. Furthermore, quantitative analyses confirm that offset-frame annotations in multicultural datasets are more uncertain, providing theoretical justification for standardizing micro-expression annotations. These findings directly support our argument for reconsidering the validity and generalizability of dataset annotation paradigms. Notably, this design can be integrated into existing models without increasing the number of parameters, offering a new approach to enhancing micro-expression recognition performance.

Research indicates that individuals who experience greater music enjoyment exhibit stronger brain–music synchronization, which is positively correlated with improved therapeutic outcomes in depression treatment. However, most existing studies on music therapy predominantly focus on emotional states, often overlooking enjoyment levels. To address this gap, we propose a novel fusion mechanism based on Fast Dynamic Time Warping (FastDTW) to objectively assess music enjoyment. First, we introduce an Adaptive Dynamic Search (ADS) algorithm that optimizes the order of Mel-Frequency Cepstral Coefficients (MFCCs) extracted from electroencephalogram (EEG) signals. This method strikes a balance between feature accuracy and redundancy, outperforming conventional fixed-order approaches. Subsequently, we analyze synchronization using the FastDTW-based Music–EEG Feature Fusion (FMEFF) framework, which leverages FastDTW to align music and EEG feature dimensions, effectively identifying the optimal alignment path to characterize synchronization. The fused features are then employed to classify music enjoyment levels, with explainability enhanced by examining classification performance across specific brain regions. Experimental results demonstrate enjoyment-level recognition accuracies of 74.50% and 81.54% in healthy and depressed participants, respectively, with the frontal lobe yielding the highest accuracy. Further correlation analysis confirms a strong relationship between the extracted features and subjective enjoyment scores. These findings present new insights for personalized music therapy recommendations and highlight the promise of integrating music and EEG signals in affective computing. This work also contributes to EEG channel optimization strategies and lays a foundation for future mental health interventions using wearable EEG systems.

Emotion classification and assessment could be a potential tool to enhance women's protection in gender-based violence situations through fear-recognition systems. Despite advances in this field, female-focused studies of emotion recognition are lacking, even if there is evidence of socialisation-related gender differences in emotion recognition and expression. To create a women-oriented EEG database focused on fear, this research explores different preprocessing, feature extraction, and selection methods, along with machine learning classifiers, to establish an effective fear recognition pipeline. The findings suggest that using a Random Forest with a 4 s window size, features selected using the MRMR algorithm, self-reported labels, and incorporating the gamma frequency band yield the optimal pipeline. To validate said results, the pipeline is tested with strict data splits (LOSO, LASO, and LOVO) as well as with a public database (SEED IV). Overall, these insights contribute to the development of EEG-based fear recognition systems specifically tailored to women's protection.

Flow, a state of complete immersion, focus, and enjoyment, has important implications for learning, productivity, and well-being. While research on flow is growing, there remains a need for refined methods to elicit and assess flow states specifically in immersive VR environments. In this work, we address this need by developing <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Beat Flow</i>, a VR music rhythm game tailored to participants' skill levels, designed to evoke distinct flow states. Instead of using traditional physiological apparatus that limits mobility, we employ lightweight wearable devices to capture electroencephalogram (EEG), galvanic skin response (GSR), photoplethysmography (PPG), and eye blink. Furthermore, we propose a VR-native self-report tool, the 3D Flow Scale (3FS), for efficient flow assessment in VR environments. Besides, we develop a deep learning model with a self-supervised pre-training strategy to classify flow states, achieving 78.4% accuracy within a 10-second time window using 5-fold cross-validation. The model demonstrates state-of-the-art performance, especially in cross-user scenarios, advancing flow detection in VR and paving the way for flow-aware VR applications across various domains.

Electroencephalogram (EEG) has been widely used for emotion recognition due to its portability and high temporal resolution. It makes success in subject-dependent scenario but faces significant challenges in cross-subject emotion recognition because of non-stationarity of EEG and individual differences. Most previous studies treat all individuals as a single source domain for transferring emotional knowledge, which may introduce irrelevant information and lead to negative transfer. Besides, there is a potential risk that some important information of common emotional features might be ignored. To deal with the issues, we propose a framework called hybrid source selection fusion domain-invariant attention (HSSFDA) for cross-subject emotion recognition. First, source domains are selected by leveraging local and global similarity for knowledge transfer. Then, a specialized attention mechanism is employed to focus on important emotional information extracted from the domain-invariant features. Finally, domain-invariant and domain-specific features are fused to enhance emotion recognition performance. To evaluate the proposed method, experiments are conducted on several public datasets including SEED, SEED_IV, DREAMER and DEAP. The results demonstrate that HSSFDA achieves accuracies of 85.07 %, 72.11 %, 62.36 %, 77.17 %, 58.51 %, and 63.55 % on SEED, SEED_IV, valence and arousal of DREAMER, and valence and arousal of DEAP datasets, respectively, demonstrating competitive performance compared to popular and state-of-the-art methods. Furthermore, we apply the HSSFDA to a self-recorded dataset collected by self-developed three-channel device and validate its effectiveness in practical applications. In conclusion, HSSFDA is a feasible method for cross-subject emotion recognition and has the potential to broaden the application of EEG in the field of affective computing.

Artificially applying specific emotions to videos of people faces with a neutral expression, while preserving the identity of the subject is a challenging task. When parts of the face are synthetically moved to generate an emotion, it typically results in spatio-temporal artifacts in the generated videos, or inconsistency to preserve the identity of subjects. Existing methods that deploy spatio-temporal convolutions and de-convolutions to generate consecutive frames in a single step are not able to ensure proper motion dynamics, in the sense that the emotion may be not visible on the face or the facial features are distorted in the video. At the same time, approaches that generate motion and identity in two separate steps are not able to ensure the consistency of the subject identity after the generation of the emotion. In this paper we propose a novel method, Video Identity-Consistent Emotion GAN (VICEGAN), that improves the video generative capabilities of two-step methods. We decouple motion and content generation, thus ensuring the consistency of subject identity in the generated videos by using an encoder-decoder generator and a new identity-preserving loss in an adversarial framework. The proposed neural network architecture also guarantees the generation of proper motion of the target expressions, mitigating the presence of artifacts. We evaluated VICEGAN on the MUG dataset and compared it with a method based on a GAN, ImaGINator, demonstrating superior performance both quantitatively and qualitatively, and with a popular method based on a diffusion model, LFDM, showing a better capability to generate recognizable emotions.

Depression is one of the most prevalent mental disorders in the world. Although some researchers have applied deep learning to assess depression, understanding its pathogenesis remains challenging. Psychological research highlights the inherent correlations among several depression symptoms. However, existing methods often ignore causal connections, treating symptoms independently and relying on local relationships without identifying valid causal paths. In this paper, we propose a novel framework, CausalSymptom, based on a variational autoencoder, to predict the severity of depression and explore its pathogenesis. The framework incorporates causal representation learning to model latent symptom states. Crucially, we provide theoretical evidence that these symptom variables can be disentangled from others by using additional information as supervision. Once these representations are learned, we present a new framework that includes a novel module, the Graph Causal Effect Network (GCEN), for the estimation of depression. This module learns a causal graph representation based on a causal total effect mechanism, which integrates causal influences across all paths, leading to a more accurate estimation of depression severity. Our model achieves competitive performance on the DAIC-WOZ and E-DAIC datasets. Furthermore, the experimental study empirically shows that our model offers insight into the pathogenesis of depression.