Emotion Recognition Using Affective Touch: A Survey

Context-aware computing has become an essential component of modern wearable technology, enabling devices to sense, interpret, and respond to users’ environments and affective states. By integrating contextual intelligence, smart systems can adapt their own behavior to better align with user needs, enhancing efficiency, personalization, and user experience. A critical aspect of context awareness is the ability to recognize a user’s internal state, particularly emotions, which influence decision-making, communication, and overall well-being. While emotion recognition is traditionally approached through multimodal sensing—using cameras, microphones, or physiological sensors such as Electroencephalography (EEG), Electrocardiography (ECG), and Electro-dermal Activity (EDA)—these methods often require multiple sensors, leading to challenges related to power consumption, complexity, and privacy. The demand for a more efficient and unobtrusive approach to emotion and context recognition has driven research toward minimal-sensor solutions that can extract meaningful affective and contextual information with reduced hardware requirements. Restricting sensing sources of contextual information by using only limited number of sensors might come at the expense of constraining the ultimate goal of context-awareness. Such expense can be mitigated by “doing more with less” i.e. delving into the opportunities of using a single sensor for multiple purposes. What is desirable is a minimal-sensor solution capable of capturing both motion and physiological signals for both robust emotion recognition and additional context estimation. Among wearable sensors, chest-worn Inertial Measurement Units (IMUs) stand out as promising candidates for comprehensive context estimation, as they are well-positioned to capture motion patterns, and physiological vibrations, being the subtle cardio-respiratory signals. This raises the question of whether a single chest-worn IMU could serve as a minimal yet effective sensor for simultaneously estimating both emotional and contextual information, offering a practical alternative to traditional multi-sensor approaches. To address the challenge of using chest-worn IMU for multiple context sensing, I investigated the potential of using Seismocardiography (SCG) and Accelerometry-Derived Respiration (ADR) to estimate emotional states. First, I explored the existing capabilities of chest-worn IMUs in capturing cardio-respiratory vibrations and motion data, which I used as a knowledge base for embedding such IMUs in context sensing. Data plays a crucial role in this problem, therefore I developed a flexible software interface, ColEmo, to facilitate the scalable collection of emotion-related data and introduced a novel dataset, EmoWear, integrating physiological and motion data for emotion recognition and context awareness. Then, I examined the feasibility of using SCG for emotion recognition, comparing its performance with traditional physiological signals and evaluating its effectiveness as a single-sensor solution. This thesis presents a detailed analysis of the potential of chest-worn IMUs for emotion and context recognition. It reports on the design and validation of a SCG-based wearable system, the development and characteristics of the EmoWear dataset, and the implementation of the ColEmo platform for emotion data collection. The research also examines machine learning techniques applied to the SCG and ADR signals, assessing their effectiveness in estimating emotional states. My findings indicate that SCG-based emotion recognition achieves performance comparable to conventional physiological counterparts, demonstrating the feasibility of using a single IMU sensor for affective computing. The publicly-available EmoWear dataset provides a valuable resource for future research, highlighting the potential of combining motion and physiological data for enhanced context awareness. Additionally, the ColEmo software interface, which I published open-source, proves to be an effective tool for multi-modal emotion data collection, facilitating further advancements in wearable emotion recognition. The results pave the way for a minimal-sensor approach for context information estimation that can significantly reduce system complexity while maintaining high performance. These findings confirm that chest-worn IMUs present an opportunity to further improve emotion recognition in addition to the motion-based context information estimation. My research highlights the need for standardized methodologies in affective computing and demonstrates the viability of motion-based physiological sensing for emotion estimation. By using a single chest-worn IMU, this work presents an alternative to multi-sensor systems, reducing power consumption, and privacy concerns while maintaining robust performance. Future research should focus on refining IMU-based emotion recognition using deep learning techniques and larger datasets to enhance classification accuracy. Further validation in real-world settings is necessary to assess the practical applicability of the proposed system. The EmoWear dataset should be explored for additional context-awareness applications, such as activity recognition and stress monitoring, to maximize its utility. The ColEmo platform offers opportunities for expansion beyond emotion recognition, including cognitive load estimation, stress monitoring, and human-computer interaction. This research moves wearable technology forward by using fewer sensors, helping make affective computing more efficient, versatile, and respectful of privacy.

The acoustic cues that convey emotion in speech are similar to those that convey emotion in music, and recognition of emotion in both of these types of cue recruits overlapping networks in the brain. Given the similarities between music and speech prosody, developmental research is uniquely positioned to determine whether recognition of these cues develops in parallel. In the present study, we asked 60 children aged 6 to 11 years, and 51 university students, to judge the emotions of 10 musical excerpts, 10 inflected speech clips, and 10 affect burst clips. We presented stimuli intended to convey happiness, sadness, anger, fear, and pride. Each emotion was presented twice per type of stimulus. We found that recognition of emotions in music and speech developed in parallel, and adult-levels of recognition develop later for these stimuli than for affect bursts. We also found that sad stimuli were most easily recognised, followed by happiness, fear, and then anger. In addition, we found that recognition of emotion in speech and affect bursts can predict emotion recognition in music stimuli independently of age and musical training. Finally, although proud speech and affect bursts were not well recognised, children aged eight years and older showed adult-like responses in recognition of proud music.

Emotion recognition from speech has emerged as an important research area in the recent past. In this regard, review of existing work on emotional speech processing is useful for carrying out further research. In this paper, the recent literature on speech emotion recognition has been presented considering the issues related to emotional speech corpora, different types of speech features and models used for recognition of emotions from speech. Thirty two representative speech databases are reviewed in this work from point of view of their language, number of speakers, number of emotions, and purpose of collection. The issues related to emotional speech databases used in emotional speech recognition are also briefly discussed. Literature on different features used in the task of emotion recognition from speech is presented. The importance of choosing different classification models has been discussed along with the review. The important issues to be considered for further emotion recognition research in general and in specific to the Indian context have been highlighted where ever necessary.

The speech signal consists of linguistic information and also paralinguistic one such as emotion. The modern automatic speech recognition systems have achieved high performance in neutral style speech recognition, but they cannot maintain their high recognition rate for spontaneous speech. So, emotion recognition is an important step toward emotional speech recognition. The accuracy of an emotion recognition system is dependent on different factors such as the type and number of emotional states and selected features, and also the type of classifier. In this paper, a modular neural-support vector machine (SVM) classifier is proposed, and its performance in emotion recognition is compared to Gaussian mixture model, multi-layer perceptron neural network, and C5.0-based classifiers. The most efficient features are also selected by using the analysis of variations method. It is noted that the proposed modular scheme is achieved through a comparative study of different features and characteristics of an individual emotional state with the aim of improving the recognition performance. Empirical results show that even by discarding 22% of features, the average emotion recognition accuracy can be improved by 2.2%. Also, the proposed modular neural-SVM classifier improves the recognition accuracy at least by 8% as compared to the simulated monolithic classifiers.

Technology-assisted intervention has the potential to adaptively individualize and improve outcomes of traditional schizophrenia SZ intervention. Virtual reality VR technology, in particular, has the potential to simulate real world social and communication interactions and hence could be useful as a therapeutic platform for SZ. Emotional face recognition is considered among the core building blocks of social communication. Studies have shown that emotional face processing and understanding is impaired in patients with SZ. The current study develops a novel VR-based system that presents avatars that can change their facial emotion dynamically for emotion recognition tasks. Additionally, this system allows real-time measurement of physiological signals and eye gaze during the emotion recognition tasks, which can be used to gain insight about the emotion recognition process in SZ population. This study further compares VR-based facial emotion recognition with that of the more traditional emotion recognition from static faces using a small usability study. Results from the usability study suggest that VR could be a viable platform for SZ intervention and implicit signals such as physiological signals and eye gaze can be utilized to better understand the underlying pattern that is not available from user reports and performance alone.

Emotion recognition using artificial intelligence (AI) has become an increasingly vital area of research, offering transformative applications across healthcare, human-computer interaction, marketing, education, and entertainment. This paper provides a comprehensive review of the techniques and applications of AI in emotion recognition. Key methodologies, including facial expression analysis, speech emotion recognition, text-based emotion detection, physiological signal analysis, and multimodal approaches, are explored in detail. Each technique's underlying algorithms, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and natural language processing (NLP) methods, are examined for their efficacy in identifying human emotions. This paper also delves into practical applications, showcasing how emotion recognition enhances mental health monitoring, improves user experience, optimizes customer service, personalizes education, and enriches entertainment media. Moreover, this study addresses the ethical and technical challenges associated with emotion recognition, such as data privacy concerns, potential biases in AI models, and the need for accuracy and reliability in emotion detection systems. The discussion extends to future directions, emphasizing the integration of AI with wearable devices, advancements in multimodal systems, and the expansion of emotion recognition technology into new domains. By presenting a thorough analysis of current techniques and applications, this paper aims to highlight the significant impact and potential of AI-driven emotion recognition, paving the way for future innovations and ethical considerations in this dynamic field. The proliferation of Internet of Things (IoT) technology has enabled the development of smart home systems that can enhance home security, energy efficiency, and convenience. This paper presents the design and implementation of an IoT-based smart home security system that utilizes a variety of sensors and actuators to monitor and control various aspects of home security. The proposed system integrates multiple components, including motion detectors, door/window sensors, cameras, smart locks, and a central control unit, all interconnected through a wireless network. The system leverages cloud computing and machine learning techniques to process sensor data, detect potential threats, and initiate appropriate responses. A user-friendly mobile application allows homeowners to monitor and control the security system remotely. The paper discusses the system architecture, hardware and software components, communication protocols, and security considerations. The implemented system demonstrates the feasibility and potential benefits of IoT-based smart home security solutions.

Emotion is a complex psychophysiological phenomenon elicited by external stimuli, exerting a profound influence on cognitive processes, decision-making, and social behavior. Emotion recognition holds broad application potential in healthcare, education, and entertainment. With virtual reality (VR) emerging as a powerful tool, it offers an immersive and controllable experimental environment. Prior studies have confirmed the feasibility and advantages of VR for emotion elicitation and recognition, and multimodal fusion has become a key strategy for enhancing recognition accuracy. However, publicly available VR multimodal emotion datasets remain limited in both scale and diversity due to the scarcity of VR content and the complexity of data collection. The shortage hampers further progress. Moreover, existing multimodal approaches still face challenges such as noise interference, large inter-individual variability, and insufficient model generalization. Achieving robust and accurate physiological signal processing and emotion modeling in VR environments thus remains an open challenge. To address the issues, we constructed a VR experimental environment and selected 10 emotion-eliciting video clips guided by the PAD(Pleasure-Arousal-Dominance) model. Thirty-eight participants (N=38) were recruited, from whom electrodermal activity, eye-tracking, and questionnaire data were collected, yielding 366 valid trials. The newly collected dataset substantially extends the publicly available VREED dataset, enriching VR-based multimodal emotion resources. Furthermore, we propose the MMTED model (Multi-Modal Temporal Emotion Detector), which incorporates baseline calibration and multimodal fusion of electrodermal and eye-tracking signals for emotion recognition. Experimental results demonstrate the strong performance of the MMTED model, achieving accuracies of 85.52% on the public VREED dataset, 89.27% on our self-collected dataset, and 85.29% on their combination.

Emotional expressions are a fundamental aspect of human communication, with speech being one of the most natural modes of interaction. Speech Emotion Recognition (SER) is a significant research topic in Natural Language Processing (NLP), aimed at identifying emotions such as satisfaction, frustration, and anger from speech audio using multiple classifiers. This article presents a method to emotion recognition from spontaneous Tunisian Dialect (TD) speech, marking the first work in the SER field to utilize spontaneous speech for emotion recognition in this dialect. The dataset was created from freely available YouTube videos across multiple domains and labeled with four perceived emotions: anger, satisfaction, frustration, and neutral. To address the data scarcity issue, we implemented data augmentation techniques, specifically Vocal Tract Length Perturbation (VTLP). The preprocessing of the speech signals involved cleaning the data from ambient and unwanted noises. We extracted and selected various spectral features, including Mel-Frequency Cepstral Coefficients (MFCCs) and Linear Prediction Cepstral Coefficients (LPCC). Subsequently, we applied several classification methods: Support Vector Machine (SVM), Bidirectional Long Short-Term Memory (BiLSTM), Long Short-Term Memory (LSTM), Convolutional Neural Network (CNN), and Random Forest. Our experiments demonstrated that the Random Forest classifier achieved the highest F-score of 58.75%. The results were thoroughly discussed, analyzed, and compared across the five models using different feature extractions. This study provides valuable insights and advancements in the SER field, particularly for the TD, for future research directions for improving emotion recognition systems.

-- Speech signals are being considered as most effective means of communication between human beings. Many researchers have found different methods or systems to identify emotions from speech signals. Here, the various features of speech are used to classify emotions. Features like pitch, tone, intensity are essential for classification. Large number of the datasets are available for speech emotion recognition. Firstly, the extraction of features from speech emotion is carried out and then another important part is classification of emotions based upon speech. Hence, different classifiers are used to classify emotions such as Happy, Sad, Anger, Surprise, Neutral, etc. Although, there are other approaches based on machine learning algorithms for identifying emotions. Speech Emotion Recognition is a current research topic because of its wide range of applications and it became a challenge in the field of speech processing too. We have carried out a brief study on Speech Emotion Analysis along with Emotion Recognition. Speech Emotion Recognition (SER) can be defined as extraction of the emotional state of the speaker from his or her speech signal. There are few universal emotions including Neutral, Anger, we have worked on different tools to be used in SER. SER is tough because emotions are subjective and annotating audio is challenging task. Emotion recognition is the part of speech recognition which is gaining more popularity and need for it increases enormously. We have classified based on different type of emotions to detect from speech. Key Words: Speech Emotion Recognition, Affective Computing, Machine Learning, Deep Learning, Audio Signal Processing, Emotion Classification, Feature Extraction, Prosodic Features, Spectral Features, Mel-Frequency Cepstral Coefficients (MFCCs), Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), Attention Mechanisms, Multimodal Emotion Recognition, Speaker-Independent SER, Real-Time Emotion Detection, Noise-Robust Emotion Recognition, Data Augmentation, Emotion-Aware Applications

Emotion recognition refers to the process of determining and identifying an individual's current emotional state by analyzing various signals such as voice, facial expressions, and physiological indicators etc. Using electroencephalogram (EEG) signals and virtual reality (VR) technology for emotion recognition research helps to better understand human emotional changes, enabling applications in areas such as psychological therapy, education, and training to enhance people's quality of life. However, there is a lack of comprehensive review literature summarizing the combined researches of EEG signals and VR environments for emotion recognition. Therefore, this paper summarizes and synthesizes relevant research from the past five years. Firstly, it introduces the relevant theories of VR and EEG signal emotion recognition. Secondly, it focuses on the analysis of emotion induction, feature extraction, and classification methods in emotion recognition using EEG signals within VR environments. The article concludes by summarizing the research's application directions and providing an outlook on future development trends, aiming to serve as a reference for researchers in related fields.

Speech Emotion Recognition (SER) remains a hot topic in the domain of affective computing, drawing considerable research interest. Its growing potential, advancements in algorithms, and real-world applications contribute to this interest. Human speech carries paralinguistic cues that can be quantitatively represented through features like pitch, intensity, and Mel-Frequency Cepstral Coefficients (MFCC). SER typically involves three main stages: data processing, feature extraction/selection, and classification based on emotional features present. These steps, tailored to the unique attributes of human speech, make machine learning methods a fitting choice for SER implementation. While recent studies in affective computing leverage various ML techniques for SER tasks, few delve into the techniques aiding these core steps. Moreover, the challenges within these stages and cutting-edge approaches addressing them often receive limited discussion or are overlooked in these works.This project introduces a pioneering Speech Emotion Recognition System leveraging BiLSTM Algorithm and Image Processing techniques, developed for implementation on the Raspberry Pi platform using Python. The system analyses recorded audio input, employing three LEDs for mood detection and interfacing with Raspberry Pi programming facilitated by an SD card. This abstract outlines an all-encompassing approach for real-time emotion recognition via audio analysis, catering to varied applications in emotional AI and human-computer interaction. KEYWORDS: Speech Emotion Recognition (SER), Machine Learning, Image Processing, Raspberry Pi.

There has been a lot of progress in recent years in the fields of expert systems, artificial intelligence (AI) and human machine interface (HMI). The use of voice commands to engage with machinery or instruct it to do a certain task is becoming more common. Numerous consumer electronics have SIRI, Alexa, Cortana, and Google Assistant built in. In the field of human-device interaction, emotion recognition from speech is a complex research subject. We can't imagine modern life without machines, so naturally there's a need to create a more robust framework for human-machine communication. A number of academics are now working on speech emotion recognition (SER) in an effort to improve the interaction between humans and machines. We aimed to identify four fundamental emotions: angry, unhappy, neutral and joyful from speech in our experiment. As you can hear below, we trained and tested our model using audio data of brief Manipuri speeches taken from films. This task makes use of convolutional neural networks (CNNs) to extract functions from speech in order to recognize different moods using the Mel-frequency cepstral coefficient (MFCC).

The history of humanity has reached Industry 4.0 that aims to the integration of information technologies and especially artificial intelligence with all life-sustaining mechanisms in the 21st century, and consecutively, the transformation of Society 5.0 has begun. Society 5.0 means a smart society in which humans share life with physical robots and software robots as well as smart devices based on augmented reality. Industry 4.0 contains main structures such as the internet of things, big data analytics, digital transformation, cyber-physical systems, artificial intelligence, and business processes optimization. It is impossible to consider the machines to be without emotions and emotional intelligence within the transformation of smart tools and artificial intelligence, in addition, while it is planned to give most of the commands with voice and speaking, it became more important to develop algorithms that can detect emotions. In the smart society, new and rapid methods are needed for speech recognition, emotion recognition, and speech emotion recognition areas to maximize human-computer (HCI) or human-robot interaction (HRI) and collaboration. In this study, speech recognition and speech emotion recognition studies in robot technology are investigated and developments are revealed.

When we are speaking with someone, the first thing we will notice about them is their face. We can easily guess their emotional state by looking at their face, such as which feeling they are experiencing. When we're talking indirectly, however, we can't make the same prediction. As a result, a human may predict the feelings of another human being to some extent. However, a machine such as a computer is incapable of doing so. When we handed up The emotional state of a human being cannot be predicted using an audio clip as input to a computer. As a result, we devised a technique dubbed "Speech Emotion Recognition"[SER] to address this issue. Different strategies can be used to address this voice emotion recognition challenge, but the accuracy is what matters. So we solved this difficulty by comparing the accuracies of two technologies. Here, The most efficient technique for voice emotion detection is the convolution neural network algorithm, which uses several modules for emotion recognition and classifiers to discriminate emotions like happy, surprise, anger, neutral mood, sadness, and so on. [IN-DEE LEARNING] Emotions play a vital role in everyone's lives. Everyone wants to know what other people are up to. As if we can say what the other person thinks when we're talking directly. I'm referring about the facial expressions as well as the tone of voice. However, when it comes to phone calls, we don't know and can't forecast what the person is thinking as an emotion, thus to fix this, we've turned to a new trend known as "Speech Emotion Recognition"[SER]. So, in order to tackle this problem, we used the Machine Learning Language, which includes an efficient algorithm known as SVM, as well as the database Ravdess[Machine Learning].

Egyptian Arabic speech emotion recognition using prosodic, spectral and wavelet features