Deep Learning-Based EEG Emotion Recognition: A Review.

Background/Objectives: The following systematic review integrates neuroimaging techniques with deep learning approaches concerning emotion detection. It, therefore, aims to merge cognitive neuroscience insights with advanced algorithmic methods in pursuit of an enhanced understanding and applications of emotion recognition. Methods: The study was conducted following PRISMA guidelines, involving a rigorous selection process that resulted in the inclusion of 64 empirical studies that explore neuroimaging modalities such as fMRI, EEG, and MEG, discussing their capabilities and limitations in emotion recognition. It further evaluates deep learning architectures, including neural networks, CNNs, and GANs, in terms of their roles in classifying emotions from various domains: human-computer interaction, mental health, marketing, and more. Ethical and practical challenges in implementing these systems are also analyzed. Results: The review identifies fMRI as a powerful but resource-intensive modality, while EEG and MEG are more accessible with high temporal resolution but limited by spatial accuracy. Deep learning models, especially CNNs and GANs, have performed well in classifying emotions, though they do not always require large and diverse datasets. Combining neuroimaging data with behavioral and cognitive features improves classification performance. However, ethical challenges, such as data privacy and bias, remain significant concerns. Conclusions: The study has emphasized the efficiencies of neuroimaging and deep learning in emotion detection, while various ethical and technical challenges were also highlighted. Future research should integrate behavioral and cognitive neuroscience advances, establish ethical guidelines, and explore innovative methods to enhance system reliability and applicability.

AI-based emotion recognition has emerged as a critical component of affect-aware educational technologies, particularly in online, large-scale, and technology-mediated learning environments where direct observation of learners’ emotional states is limited. By leveraging facial expressions, speech, physiological signals, and interaction data, emotion-aware systems aim to support adaptive instruction, learner engagement, and instructional decision-making. Despite rapid technical progress, the integration of emotion recognition into authentic educational practice remains constrained by dataset limitations, cultural bias, weak pedagogical alignment, and unresolved ethical concerns. This article presents a critical and theory-informed review of recent advances in AI-based emotion recognition in education, with particular emphasis on deep learning architectures and multimodal fusion approaches, while highlighting persistent challenges and future research directions.Purpose:This study aims to critically examine the current state of AI-based emotion recognition in educational contexts by analyzing dominant modeling paradigms, sensing modalities, application domains, and ethical implications, in order to identify key gaps between technical development and pedagogical practice.Methods/Study design/approach:The study adopts a narrative literature review approach grounded in educational and affective computing theories. Representative and influential studies were analyzed across facial, speech, physiological, and multimodal emotion recognition systems, with emphasis on deep learning, attention-based, and transformer architectures. The review synthesizes findings related to system performance, educational applications, dataset characteristics, and ethical considerations.Result/Findings:The findings indicate that multimodal emotion recognition systems generally outperform unimodal approaches in detecting learning-relevant affective states such as engagement, frustration, and confusion. Advances in deep learning, particularly attention-based and transformer models, improve temporal modeling and robustness. However, most systems rely on non-educational datasets, demonstrate limited cultural generalizability, and prioritize recognition accuracy over pedagogical interpretability and long-term educational impact. Ethical challenges related to privacy, consent, bias, and emotional surveillance remain insufficiently addressed.Novelty/Originality/Value:This study provides an integrative and pedagogically grounded synthesis that bridges affective computing research with educational theory and governance perspectives. By reframing AI-based emotion recognition as a decision-support mechanism rather than an automated control system, the article offers a conceptual foundation for responsible, culturally inclusive, and ethically aligned deployment of emotion-aware AI in education, serving as a reference for future interdisciplinary research and policy development.

Emotion detection, hence, is an important step toward making human-computer interaction a more enhanced process, where systems are made capable of identifying and responding to the emotional state of users. In fact, multimodal emotion detection systems in which both auditory and visual information are fused are emerging, and these approaches toward expressive emotional states are complementary and robust. Multimodal systems enhance the quality of interacting and, through many applications, can diagnose emotional disorders, monitor automotive safety, and improve human-robot interactions. In nature, the high-dimensional space and dynamic threats have resulted in obtaining low accuracy and high computational cost in applying the traditional models based on single-modality data. On the other hand, multimodal systems explore the synergy between audio and visual data, giving better performance and higher accuracy in inferring subtle emotional expressions. The latest improvement was done on these systems using recent advancements in transfer learning and deep learning techniques.That being said, this research Proposal devises a multimodal emotion recognition system integrating speech and face information through transfer learning for improved accuracy and robustness. Serving this purpose, the objectives of this research entail the effective comparison among different transfer-learning strategies, including the impact of pre-trained models in speech-based emotion recognition, and to introduce the role of voice activity detection in the process. Advanced neural network architectures like Spatial Transformer Networks and bidirectional LSTM in facial emotion recognition will also be tested. Early and late fusion strategies will also be used to find the best strategy for combining speech and facial data.This research will target several challenges that involve the complexity of data, balancing of the model performance-robustness balance, computational limitations, and standardization of evaluations in developing a working and robust emotion recognition system to enhance digital interaction and apply in practical areas. The goal is to create a system that oversteps the limitation of single-modality models through state-of-the-art advances in deep learning, as well as front-line improvements in transfer learning, in the manner of emotion detection performance.

Context-aware computing has become an essential component of modern wearable technology, enabling devices to sense, interpret, and respond to users’ environments and affective states. By integrating contextual intelligence, smart systems can adapt their own behavior to better align with user needs, enhancing efficiency, personalization, and user experience. A critical aspect of context awareness is the ability to recognize a user’s internal state, particularly emotions, which influence decision-making, communication, and overall well-being. While emotion recognition is traditionally approached through multimodal sensing—using cameras, microphones, or physiological sensors such as Electroencephalography (EEG), Electrocardiography (ECG), and Electro-dermal Activity (EDA)—these methods often require multiple sensors, leading to challenges related to power consumption, complexity, and privacy. The demand for a more efficient and unobtrusive approach to emotion and context recognition has driven research toward minimal-sensor solutions that can extract meaningful affective and contextual information with reduced hardware requirements. Restricting sensing sources of contextual information by using only limited number of sensors might come at the expense of constraining the ultimate goal of context-awareness. Such expense can be mitigated by “doing more with less” i.e. delving into the opportunities of using a single sensor for multiple purposes. What is desirable is a minimal-sensor solution capable of capturing both motion and physiological signals for both robust emotion recognition and additional context estimation. Among wearable sensors, chest-worn Inertial Measurement Units (IMUs) stand out as promising candidates for comprehensive context estimation, as they are well-positioned to capture motion patterns, and physiological vibrations, being the subtle cardio-respiratory signals. This raises the question of whether a single chest-worn IMU could serve as a minimal yet effective sensor for simultaneously estimating both emotional and contextual information, offering a practical alternative to traditional multi-sensor approaches. To address the challenge of using chest-worn IMU for multiple context sensing, I investigated the potential of using Seismocardiography (SCG) and Accelerometry-Derived Respiration (ADR) to estimate emotional states. First, I explored the existing capabilities of chest-worn IMUs in capturing cardio-respiratory vibrations and motion data, which I used as a knowledge base for embedding such IMUs in context sensing. Data plays a crucial role in this problem, therefore I developed a flexible software interface, ColEmo, to facilitate the scalable collection of emotion-related data and introduced a novel dataset, EmoWear, integrating physiological and motion data for emotion recognition and context awareness. Then, I examined the feasibility of using SCG for emotion recognition, comparing its performance with traditional physiological signals and evaluating its effectiveness as a single-sensor solution. This thesis presents a detailed analysis of the potential of chest-worn IMUs for emotion and context recognition. It reports on the design and validation of a SCG-based wearable system, the development and characteristics of the EmoWear dataset, and the implementation of the ColEmo platform for emotion data collection. The research also examines machine learning techniques applied to the SCG and ADR signals, assessing their effectiveness in estimating emotional states. My findings indicate that SCG-based emotion recognition achieves performance comparable to conventional physiological counterparts, demonstrating the feasibility of using a single IMU sensor for affective computing. The publicly-available EmoWear dataset provides a valuable resource for future research, highlighting the potential of combining motion and physiological data for enhanced context awareness. Additionally, the ColEmo software interface, which I published open-source, proves to be an effective tool for multi-modal emotion data collection, facilitating further advancements in wearable emotion recognition. The results pave the way for a minimal-sensor approach for context information estimation that can significantly reduce system complexity while maintaining high performance. These findings confirm that chest-worn IMUs present an opportunity to further improve emotion recognition in addition to the motion-based context information estimation. My research highlights the need for standardized methodologies in affective computing and demonstrates the viability of motion-based physiological sensing for emotion estimation. By using a single chest-worn IMU, this work presents an alternative to multi-sensor systems, reducing power consumption, and privacy concerns while maintaining robust performance. Future research should focus on refining IMU-based emotion recognition using deep learning techniques and larger datasets to enhance classification accuracy. Further validation in real-world settings is necessary to assess the practical applicability of the proposed system. The EmoWear dataset should be explored for additional context-awareness applications, such as activity recognition and stress monitoring, to maximize its utility. The ColEmo platform offers opportunities for expansion beyond emotion recognition, including cognitive load estimation, stress monitoring, and human-computer interaction. This research moves wearable technology forward by using fewer sensors, helping make affective computing more efficient, versatile, and respectful of privacy.

OBJECTIVE This systematic review provides a synthesis of the existing data concerning the Electroencephalogram (EEG)-based emotion recognition and assesses the development of the old machine learning models to the current deep learning models. The purpose of the review is the comparison of their performance and the identification of trends in the approaches to the methodology and the evaluation of the strength and the reproducibility of the discipline. METHODS The review was done based on the Preferred Reporting Items of Systematic Reviews and Meta-Analyses (PRISMA 2020) guidelines. Five electronic databases (IEEE Xplore, Scopus, PubMed, ScienceDirect, and SpringerLink) that have been published not earlier than January 2012 were searched systematically. Due to the removal of duplicates and two rounds of screening against pre-defined inclusion criteria, 50 studies were incorporated to be final synthesized. FINDINGS It has been demonstrated that there is a definite trend towards end-to-end deep learning models, especially Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) networks, and architectures combining both. These models, particularly when using maps of topographic features and maps of functional connectivity, have shown a higher level of performance, and their classification accuracies of 90 percent or higher in benchmark data such as DEAP and SEED in subject-dependent contexts. But, there is a significant decline in the performance in cross-subject validation, which is an outstanding generalization issue. It also becomes evident during the synthesis that validation protocols, data preprocessing and reporting standards exhibit high heterogeneity, thus making it difficult to directly compare them and jeopardizing reproducibility. CONCLUSION Deep learning approaches are an important development in emotion recognition of EEG, but the area is plagued by lack of uniformity and focus on real-world applicability. The next step in work is to focus on the creation of standardized evaluation metrics, explicable AI methods, and effective, cross-subject models to enable the movement of laboratory studies to the reliable, deployable systems.

Emotion recognition from EEG signals plays a significant role in affective computing, and it enables the systems to be aware of the human emotional state. This research proposes a novel methodology for zero-shot learning in multi-class emotion recognition from EEG signals, which address the issues like data scarcity, class imbalance, and generalization. A technique starting with a dataset acquisition method makes use of EEG-GANs in order to produce synthetic samples to balance class imbalance for poorly represented emotional classes. The introduction of DWC-TFS is innovatively used: the dynamic patterns in connectivity have been captured appropriately and the temporally noisy has reduced in the study. In order to extract features, DFCNs are applied in modeling time-varying relationships between EEG channels, keeping important temporal dependencies intact. For feature selection, Neuro-Symbolic Feature Selection is used by incorporating symbolic reasoning with deep learning in order to choose the most discriminative features. The classification task is done using a T-ZSC Transformer-Based Zero-Shot Classifier. The model will generalize across unseen emotional classes through the use of semantic embeddings and self-attention mechanisms. The proposed methodology achieves an accuracy of 96.2%, demonstrating robust performance in emotion recognition. This integrated approach offers a powerful solution for addressing challenges in EEG-based emotion recognition, ensuring improved class balance, generalization, and accuracy in real-world applications.

Emotion recognition (ER) is crucial for understanding human behaviours, social interactions, and psychological well-being. Electroencephalography (EEG) has emerged as a promising tool for capturing the neural correlates of emotions. This work is a systematic review of articles in ER using EEG signals. A total of 120 articles from 1041 articles were selected based on PRISMA guidelines using defined inclusion and exclusion criteria, published between 2018 and 2024. This article aims to provide an in-depth understanding of the current landscape of ER from EEG signals utilizing deep learning (DL). This review offers valuable guidance for researchers and practitioners seeking more refined and reliable emotion classification systems. To explore the effectiveness of DL models in EEG-based ER, several potential DL models, such as convolutional neural network, long short-term memory (LSTM), gated recurrent unit (GRU), hybrid bidirectional LSTM (BiLSTM), bidirectional GRU, and advanced DL models such as convolutional recurrent neural network and EEG-Conformer models are applied to two popular datasets, SEED and GAMEEMO, respectively, to depict the full process of ER. Additionally, the performance of DL models is also compared with the performance of basic machine learning (ML) models such as SVM, k-nearest neighbors, logistic regression, and boosting algorithms such as AdaBoost, XGBoost and LightGBM. Through extensive experiments and performance evaluations, the performance of different models when applied to the datasets mentioned above is compared. The accuracy, precision, recall, and F1-scores are analysed to determine the most effective model for EEG-based ER. The findings of this study demonstrate that the performance of hybrid DL models is more efficacious than that of ML models. The best-performing model (BiLSTM) classified the emotions, with an accuracy of 90.54% when applied to the GAMEEMO dataset. This research contributes to the growing body of literature on ER and provides insights into the feasibility of using EEG signals to understand emotional states, and presents a structured roadmap for future exploration. The findings can aid in the development of more accurate and reliable ER systems, which can have wide-ranging applications in psychology, social sciences, and human-computer interactions.

Emotion recognition using electroencephalography (EEG) signals has gained significant attention in affective computing, human-computer interaction, and mental health applications. Owing to the complex and non-stationary nature of EEG signals, diverse feature extraction methods - spanning time-domain (statistical), frequency-domain, and deep learning approaches - have been explored to capture intrinsic signal characteristics. This study presents a critical evaluation of three machine learning models for EEG-based emotion classification, leveraging both time- and frequency-domain features. Specifically, the performance of eXtreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LightGBM), and K-Nearest Neighbors (KNN) was investigated using the DEAP (Database for Emotion Analysis using Physiological Signals) dataset, a widely adopted benchmark for emotion recognition research. To enhance signal quality, EEG data were segmented into 1-second epochs, from which a comprehensive set of statistical and mathematical descriptors were extracted: mean, standard deviation, root mean square, minimum, maximum, peak value, activity, mobility, complexity, entropy, skewness, and kurtosis in the time-domain, along with entropy in the frequency-domain. A 10-fold cross-validation procedure was applied under a cross-subject evaluation scheme to assess generalization across individuals. Experimental results demonstrated that KNN achieved the highest classification accuracy (96.77%), surpassing XGBoost (88.55%) and LightGBM (84.08%), thereby challenging the conventional preference for more complex ensemble models in EEG-based emotion recognition. These findings highlight the critical role of feature engineering and classifier selection in optimizing system performance. The outcomes have practical implications in clinical diagnostics (e.g., early detection of depression and anxiety) and adaptive systems, including mood-aware interfaces, neurofeedback applications, and emotionally adaptive gaming environments.

The acoustic cues that convey emotion in speech are similar to those that convey emotion in music, and recognition of emotion in both of these types of cue recruits overlapping networks in the brain. Given the similarities between music and speech prosody, developmental research is uniquely positioned to determine whether recognition of these cues develops in parallel. In the present study, we asked 60 children aged 6 to 11 years, and 51 university students, to judge the emotions of 10 musical excerpts, 10 inflected speech clips, and 10 affect burst clips. We presented stimuli intended to convey happiness, sadness, anger, fear, and pride. Each emotion was presented twice per type of stimulus. We found that recognition of emotions in music and speech developed in parallel, and adult-levels of recognition develop later for these stimuli than for affect bursts. We also found that sad stimuli were most easily recognised, followed by happiness, fear, and then anger. In addition, we found that recognition of emotion in speech and affect bursts can predict emotion recognition in music stimuli independently of age and musical training. Finally, although proud speech and affect bursts were not well recognised, children aged eight years and older showed adult-like responses in recognition of proud music.

-- Speech signals are being considered as most effective means of communication between human beings. Many researchers have found different methods or systems to identify emotions from speech signals. Here, the various features of speech are used to classify emotions. Features like pitch, tone, intensity are essential for classification. Large number of the datasets are available for speech emotion recognition. Firstly, the extraction of features from speech emotion is carried out and then another important part is classification of emotions based upon speech. Hence, different classifiers are used to classify emotions such as Happy, Sad, Anger, Surprise, Neutral, etc. Although, there are other approaches based on machine learning algorithms for identifying emotions. Speech Emotion Recognition is a current research topic because of its wide range of applications and it became a challenge in the field of speech processing too. We have carried out a brief study on Speech Emotion Analysis along with Emotion Recognition. Speech Emotion Recognition (SER) can be defined as extraction of the emotional state of the speaker from his or her speech signal. There are few universal emotions including Neutral, Anger, we have worked on different tools to be used in SER. SER is tough because emotions are subjective and annotating audio is challenging task. Emotion recognition is the part of speech recognition which is gaining more popularity and need for it increases enormously. We have classified based on different type of emotions to detect from speech. Key Words: Speech Emotion Recognition, Affective Computing, Machine Learning, Deep Learning, Audio Signal Processing, Emotion Classification, Feature Extraction, Prosodic Features, Spectral Features, Mel-Frequency Cepstral Coefficients (MFCCs), Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), Attention Mechanisms, Multimodal Emotion Recognition, Speaker-Independent SER, Real-Time Emotion Detection, Noise-Robust Emotion Recognition, Data Augmentation, Emotion-Aware Applications

EEG-based emotion recognition: A deep learning approach to brain region analysis

When we are speaking with someone, the first thing we will notice about them is their face. We can easily guess their emotional state by looking at their face, such as which feeling they are experiencing. When we're talking indirectly, however, we can't make the same prediction. As a result, a human may predict the feelings of another human being to some extent. However, a machine such as a computer is incapable of doing so. When we handed up The emotional state of a human being cannot be predicted using an audio clip as input to a computer. As a result, we devised a technique dubbed "Speech Emotion Recognition"[SER] to address this issue. Different strategies can be used to address this voice emotion recognition challenge, but the accuracy is what matters. So we solved this difficulty by comparing the accuracies of two technologies. Here, The most efficient technique for voice emotion detection is the convolution neural network algorithm, which uses several modules for emotion recognition and classifiers to discriminate emotions like happy, surprise, anger, neutral mood, sadness, and so on. [IN-DEE LEARNING] Emotions play a vital role in everyone's lives. Everyone wants to know what other people are up to. As if we can say what the other person thinks when we're talking directly. I'm referring about the facial expressions as well as the tone of voice. However, when it comes to phone calls, we don't know and can't forecast what the person is thinking as an emotion, thus to fix this, we've turned to a new trend known as "Speech Emotion Recognition"[SER]. So, in order to tackle this problem, we used the Machine Learning Language, which includes an efficient algorithm known as SVM, as well as the database Ravdess[Machine Learning].

Emotion recognition from speech has emerged as an important research area in the recent past. In this regard, review of existing work on emotional speech processing is useful for carrying out further research. In this paper, the recent literature on speech emotion recognition has been presented considering the issues related to emotional speech corpora, different types of speech features and models used for recognition of emotions from speech. Thirty two representative speech databases are reviewed in this work from point of view of their language, number of speakers, number of emotions, and purpose of collection. The issues related to emotional speech databases used in emotional speech recognition are also briefly discussed. Literature on different features used in the task of emotion recognition from speech is presented. The importance of choosing different classification models has been discussed along with the review. The important issues to be considered for further emotion recognition research in general and in specific to the Indian context have been highlighted where ever necessary.

The speech signal consists of linguistic information and also paralinguistic one such as emotion. The modern automatic speech recognition systems have achieved high performance in neutral style speech recognition, but they cannot maintain their high recognition rate for spontaneous speech. So, emotion recognition is an important step toward emotional speech recognition. The accuracy of an emotion recognition system is dependent on different factors such as the type and number of emotional states and selected features, and also the type of classifier. In this paper, a modular neural-support vector machine (SVM) classifier is proposed, and its performance in emotion recognition is compared to Gaussian mixture model, multi-layer perceptron neural network, and C5.0-based classifiers. The most efficient features are also selected by using the analysis of variations method. It is noted that the proposed modular scheme is achieved through a comparative study of different features and characteristics of an individual emotional state with the aim of improving the recognition performance. Empirical results show that even by discarding 22% of features, the average emotion recognition accuracy can be improved by 2.2%. Also, the proposed modular neural-SVM classifier improves the recognition accuracy at least by 8% as compared to the simulated monolithic classifiers.

Emotion recognition based on EEG signals is a critical area within affective computing, with applications in mental health monitoring, human-computer interaction, and neuroadaptive systems. However, accurately classifying emotional states from inherently non-stationary and noisy EEG data remains a major challenge. This study explores the classification of three discrete emotions, Humorous, Sad, and Fearful, elicited through video stimuli, using EEG recordings from six participants acquired via a 19-channel Mitsar amplifier at a 500 Hz sampling rate. Preprocessing steps included bandpass filtering (1–40 Hz), epoch segmentation, and multi-domain feature extraction encompassing statistical measures, spectral features, differential entropy, Hjorth parameters, and hemispheric asymmetry indicators. Data augmentation was applied to balance class distributions, particularly for the underrepresented fear category. The resulting features were normalized and structured to support temporal deep learning and classical machine learning models. The classification performance of Long Short-Term Memory (LSTM) networks was evaluated alongside Support Vector Machine (SVM), k-Nearest Neighbors (k-NN), and Random Forest (RF) classifiers. While LSTM demonstrated competency in capturing temporal dependencies, especially in fear recognition, SVM achieved the highest overall accuracy, 94.12%, outperforming LSTM at 85.16%, RF at 90.00%, and k-NN at 78.01%. These results suggest that when robust and discriminative features are employed, traditional models like SVM can surpass deep learning methods, particularly in small-scale EEG datasets with limited temporal complexity. This study underscores the importance of aligning model architecture with feature representation and contributes a comparative evaluation framework for EEG-based emotion recognition systems.