Speech emotion recognition using both deep learning and machine learning techniques

Context-aware computing has become an essential component of modern wearable technology, enabling devices to sense, interpret, and respond to users’ environments and affective states. By integrating contextual intelligence, smart systems can adapt their own behavior to better align with user needs, enhancing efficiency, personalization, and user experience. A critical aspect of context awareness is the ability to recognize a user’s internal state, particularly emotions, which influence decision-making, communication, and overall well-being. While emotion recognition is traditionally approached through multimodal sensing—using cameras, microphones, or physiological sensors such as Electroencephalography (EEG), Electrocardiography (ECG), and Electro-dermal Activity (EDA)—these methods often require multiple sensors, leading to challenges related to power consumption, complexity, and privacy. The demand for a more efficient and unobtrusive approach to emotion and context recognition has driven research toward minimal-sensor solutions that can extract meaningful affective and contextual information with reduced hardware requirements. Restricting sensing sources of contextual information by using only limited number of sensors might come at the expense of constraining the ultimate goal of context-awareness. Such expense can be mitigated by “doing more with less” i.e. delving into the opportunities of using a single sensor for multiple purposes. What is desirable is a minimal-sensor solution capable of capturing both motion and physiological signals for both robust emotion recognition and additional context estimation. Among wearable sensors, chest-worn Inertial Measurement Units (IMUs) stand out as promising candidates for comprehensive context estimation, as they are well-positioned to capture motion patterns, and physiological vibrations, being the subtle cardio-respiratory signals. This raises the question of whether a single chest-worn IMU could serve as a minimal yet effective sensor for simultaneously estimating both emotional and contextual information, offering a practical alternative to traditional multi-sensor approaches. To address the challenge of using chest-worn IMU for multiple context sensing, I investigated the potential of using Seismocardiography (SCG) and Accelerometry-Derived Respiration (ADR) to estimate emotional states. First, I explored the existing capabilities of chest-worn IMUs in capturing cardio-respiratory vibrations and motion data, which I used as a knowledge base for embedding such IMUs in context sensing. Data plays a crucial role in this problem, therefore I developed a flexible software interface, ColEmo, to facilitate the scalable collection of emotion-related data and introduced a novel dataset, EmoWear, integrating physiological and motion data for emotion recognition and context awareness. Then, I examined the feasibility of using SCG for emotion recognition, comparing its performance with traditional physiological signals and evaluating its effectiveness as a single-sensor solution. This thesis presents a detailed analysis of the potential of chest-worn IMUs for emotion and context recognition. It reports on the design and validation of a SCG-based wearable system, the development and characteristics of the EmoWear dataset, and the implementation of the ColEmo platform for emotion data collection. The research also examines machine learning techniques applied to the SCG and ADR signals, assessing their effectiveness in estimating emotional states. My findings indicate that SCG-based emotion recognition achieves performance comparable to conventional physiological counterparts, demonstrating the feasibility of using a single IMU sensor for affective computing. The publicly-available EmoWear dataset provides a valuable resource for future research, highlighting the potential of combining motion and physiological data for enhanced context awareness. Additionally, the ColEmo software interface, which I published open-source, proves to be an effective tool for multi-modal emotion data collection, facilitating further advancements in wearable emotion recognition. The results pave the way for a minimal-sensor approach for context information estimation that can significantly reduce system complexity while maintaining high performance. These findings confirm that chest-worn IMUs present an opportunity to further improve emotion recognition in addition to the motion-based context information estimation. My research highlights the need for standardized methodologies in affective computing and demonstrates the viability of motion-based physiological sensing for emotion estimation. By using a single chest-worn IMU, this work presents an alternative to multi-sensor systems, reducing power consumption, and privacy concerns while maintaining robust performance. Future research should focus on refining IMU-based emotion recognition using deep learning techniques and larger datasets to enhance classification accuracy. Further validation in real-world settings is necessary to assess the practical applicability of the proposed system. The EmoWear dataset should be explored for additional context-awareness applications, such as activity recognition and stress monitoring, to maximize its utility. The ColEmo platform offers opportunities for expansion beyond emotion recognition, including cognitive load estimation, stress monitoring, and human-computer interaction. This research moves wearable technology forward by using fewer sensors, helping make affective computing more efficient, versatile, and respectful of privacy.

Emotion recognition plays a pivotal role in advancing Human-Computer Interaction (HCI) by enabling systems to understand and respond to human emotional states. Among various modalities, Speech Emotion Recognition (SER) stands out as a non-invasive, cost-effective, and temporally efficient approach for detecting emotions. This study leverages the RAVDESS dataset, which provides high-quality audio recordings for emotion classification. The proposed methodology involves preprocessing audio signals to remove noise, extracting temporal and spectral features in both time and frequency domains, and implementing machine learning models for multi-class emotion classification. The study evaluates and compares the performance of several classification models, including Random Forest (RF), Multilayer Perceptron (MLP), Support Vector Machine (SVM), Convolutional Neural Networks (CNN), and Decision Trees (DT). Experimental results demonstrate promising accuracy levels, highlighting the potential of these machine learning techniques in SER. This research contributes to the integration of SER within Brain-Computer Interfaces (BCI) and other emotion-aware applications, paving the way for enhanced interactive systems. Keywords – Speech Emotion Recognition -Machine Learning - Random Forest -Multilayer Perceptron - Support Vector Machine - Convolutional Neural Networks - Decision Tree - RAVDESS Dataset

Emotion recognition from speech plays a crucial role in enhancing human–computer interaction by enabling systems to interpret and respond to users’ emotional states. This study develops and evaluates a Speech Emotion Recognition (SER) system using three machine learning techniques; Support Vector Machines (SVM), Multilayer Perceptron (MLP), and Convolutional Neural Networks (CNNs). The system is trained and tested on the RAVDESS dataset, which contains 1,440 professionally recorded audio samples representing a wide range of emotions. Our approach involves careful preprocessing of the audio signals, extraction of key acoustic features, and comparative performance evaluation of the three models using standard metrics. Results show that each model exhibits unique strengths and limitations, with CNNs achieving the most robust feature learning and generalization. The study underscores the importance of diverse feature representation for accurate emotion classification and provides insight into how different model architectures handle emotional nuances in speech. Identified challenges such as dataset diversity, feature selection, and computational complexity are discussed, along with recommendations for future research to improve SER systems’ real-world adaptability. This work contributes to ongoing efforts toward developing emotionally aware technologies that can enhance natural human–machine communication.

-- Speech signals are being considered as most effective means of communication between human beings. Many researchers have found different methods or systems to identify emotions from speech signals. Here, the various features of speech are used to classify emotions. Features like pitch, tone, intensity are essential for classification. Large number of the datasets are available for speech emotion recognition. Firstly, the extraction of features from speech emotion is carried out and then another important part is classification of emotions based upon speech. Hence, different classifiers are used to classify emotions such as Happy, Sad, Anger, Surprise, Neutral, etc. Although, there are other approaches based on machine learning algorithms for identifying emotions. Speech Emotion Recognition is a current research topic because of its wide range of applications and it became a challenge in the field of speech processing too. We have carried out a brief study on Speech Emotion Analysis along with Emotion Recognition. Speech Emotion Recognition (SER) can be defined as extraction of the emotional state of the speaker from his or her speech signal. There are few universal emotions including Neutral, Anger, we have worked on different tools to be used in SER. SER is tough because emotions are subjective and annotating audio is challenging task. Emotion recognition is the part of speech recognition which is gaining more popularity and need for it increases enormously. We have classified based on different type of emotions to detect from speech. Key Words: Speech Emotion Recognition, Affective Computing, Machine Learning, Deep Learning, Audio Signal Processing, Emotion Classification, Feature Extraction, Prosodic Features, Spectral Features, Mel-Frequency Cepstral Coefficients (MFCCs), Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), Attention Mechanisms, Multimodal Emotion Recognition, Speaker-Independent SER, Real-Time Emotion Detection, Noise-Robust Emotion Recognition, Data Augmentation, Emotion-Aware Applications

Growth in system complexity increases the need for automated techniques dedicated to different log analysis tasks such as Log-based Anomaly Detection (LAD). The latter has been widely addressed in the literature, mostly by means of a variety of deep learning techniques. However, despite their many advantages, that focus on deep learning techniques is somewhat arbitrary as traditional Machine Learning (ML) techniques may perform well in many cases, depending on the context and datasets. In the same vein, semi-supervised techniques deserve the same attention as supervised techniques since the former have clear practical advantages. Further, current evaluations mostly rely on the assessment of detection accuracy. However, this is not enough to decide whether or not a specific ML technique is suitable to address the LAD problem in a given context. Other aspects to consider include training and prediction times as well as the sensitivity to hyperparameter tuning, which in practice matters to engineers. In this paper, we present a comprehensive empirical study, in which we evaluate a wide array of supervised and semi-supervised, traditional and deep ML techniques w.r.t. four evaluation criteria: detection accuracy, time performance, sensitivity of detection accuracy and time performance to hyperparameter tuning. Our goal is to provide much stronger and comprehensive evidence regarding the relative advantages and drawbacks of alternative techniques for LAD. The experimental results show that supervised traditional and deep ML techniques fare similarly in terms of their detection accuracy and prediction time on most of the benchmark datasets considered in our study. Moreover, overall, sensitivity analysis to hyperparameter tuning with respect to detection accuracy shows that supervised traditional ML techniques are less sensitive than deep learning techniques. Further, semi-supervised techniques yield significantly worse detection accuracy than supervised techniques.

Agriculture has been a vital sector for the majority of people, especially in countries like India. However, the increasing need for food production has led to intensive farming practices that have resulted in the deterioration of soil quality. This deterioration in soil quality poses significant challenges to both agricultural productivity and environmental sustainability. To address these challenges, advanced soil nutrient prediction systems that utilize machine learning and deep learning techniques are being developed. These advanced soil nutrient prediction systems utilize various sources of data, such as soil parameters, plant diseases, pests, fertilizer usage, and changes in weather patterns. By mapping and analyzing these data sources, machine learning algorithms can accurately predict the distribution of soil nutrients and other properties essential for precise agricultural practices. A previous study compared machine learning algorithms like SVM and Random Forest with deep learning algorithms CNN and LSTM for predicting crop yields. The most appropriate model is a significant challenge, but several studies have evaluated recommendation system models using deep machine learning techniques. Deep learning models attain accuracy above 90%, while many ML models achieve rates between 90% and 93%. Furthermore, the research seeks to propose specific crop suggestions grounded in soil nutrients for precision agriculture to enhance crop productivity.

The acoustic cues that convey emotion in speech are similar to those that convey emotion in music, and recognition of emotion in both of these types of cue recruits overlapping networks in the brain. Given the similarities between music and speech prosody, developmental research is uniquely positioned to determine whether recognition of these cues develops in parallel. In the present study, we asked 60 children aged 6 to 11 years, and 51 university students, to judge the emotions of 10 musical excerpts, 10 inflected speech clips, and 10 affect burst clips. We presented stimuli intended to convey happiness, sadness, anger, fear, and pride. Each emotion was presented twice per type of stimulus. We found that recognition of emotions in music and speech developed in parallel, and adult-levels of recognition develop later for these stimuli than for affect bursts. We also found that sad stimuli were most easily recognised, followed by happiness, fear, and then anger. In addition, we found that recognition of emotion in speech and affect bursts can predict emotion recognition in music stimuli independently of age and musical training. Finally, although proud speech and affect bursts were not well recognised, children aged eight years and older showed adult-like responses in recognition of proud music.

A comprehensive review on detection of plant disease using machine learning and deep learning approaches

Coastal aquifers are vital for sustaining ecosystems and providing freshwater for agriculture and domestic use. However, rising groundwater salinity in these regions demands innovative approaches for accurate modeling and prediction. In recent decades, machine learning (ML) and deep learning (DL) techniques have been extensively used across various water resource management fields, often yielding promising outcomes. This review examines the growing usage of ML and DL techniques in modeling groundwater salinity in coastal aquifers. The analysis is based on 104 peer-reviewed journal articles including the review articles indexed in PubMed, ScienceDirect, Google Scholar, Scopus, and SpringerLink using keywords like "saltwater intrusion", "machine learning", “deep learning”, and "groundwater salinity". The review discusses recent advancements, challenges, and future directions for improving ML model accuracy and applicability. Studies reveal an increasing reliance on ML techniques to predict groundwater salinity and manage seawater intrusion across various coastal geological settings. Researchers have employed a range of ML models, from traditional methods like Artificial Neural Networks and Adaptive Neuro Fuzzy Inference Systems to advanced DL-based models like Convolutional Neural Networks and Long Short-Term Memory Networks, often using ensemble methods to enhance accuracy. Key findings include the top-tier output of ensemble and optimized models, including Random Forest and Grasshopper Optimization Algorithm-XGBoost. These studies highlight the importance of model selection, input variable ranking, and computational efficiency, demonstrating ML's global relevance in environmental management. The review highlights the prospects of ML, DL, and ensemble models to redefine groundwater management in coastal aquifers, offering more effective and sustainable solutions to the challenges posed by groundwater salinity and saltwater intrusion. Future advancements will focus on further integrating DL, explainable artificial intelligence, and ensemble approaches, along with other innovative techniques, to explore sparsely studied variables, model new or unique study areas, and apply ML methods for managing groundwater salinity.

This review offers an in-depth examination of Deep Learning (DL) and Machine Learning (ML) techniques for smart grid load forecasting, emphasizing language precision, methodological rigor, and the exploration of novel contributions. The language used in this review is both technical and accessible, balancing complex concepts with clear explanations to cater to both specialists and general readers. It meticulously dissects contemporary DL models, including neural networks and ensemble methods, and evaluates their effectiveness through a detailed review of algorithms and frameworks. The methodology section systematically compares these techniques against traditional forecasting methods using performance metrics such as MAPE, RMSE, and MSE, ensuring a comprehensive assessment of their accuracy and scalability. A significant contribution of this review is its examination of real-world applications and case studies, which demonstrate how ML and DL techniques address practical challenges in energy management, such as grid stability and demand forecasting. Furthermore, the review introduces novel perspectives on the integration of probabilistic forecasting and ensemble methods, which offer innovative approaches for managing energy demand uncertainties. By identifying current limitations and proposing future research directions, this review not only advances the understanding of DL and ML applications in smart grids but also provides a foundation for future developments in this evolving field.

Emotions are ubiquitous to human life. Previous studies suggest that ninety percent of the life experience of humankind is affected by at least one emotion, and that emotional state guides their thoughts and behavior. The neurological study suggests that the design of human brain ensures that the emotional brain almost always influences the rational brain. Which means every rational decision is affected by the underlying emotional state. For decades, the influence of emotional states on human life has been overlooked by the AI researchers until recently. For the last twenty years, the study on different aspects of emotions and its detection for the more significant cause is gaining considerable attention and importance. With the advent of the Fourth Industrial Revolution (4IR), advancements in Artificial Intelligence (AI) is already touching new heights and in this context successful detection of emotional states by AI systems will be another big leap towards more realistic social AI agents. This chapter will discuss in detail the importance and recent technological advancements towards emotion recognition (ER) in the backdrop of the Fourth Industrial Revolution. ER uses different types of human physical and behavioral expressions, such as facial features, speech features, and their variants. Deep Learning (DL) has emerged as one of the prominent pillars of 4IR. ER task extensively uses DL techniques for classifying different emotions. So, this chapter includes a discussion of relevant technical aspects of DL. Finally, it includes a brief description of a newly proposed Speech Emotion Recognition (SER) model as a case study of the ER systems.

Breast cancer is one of the most prevalent and chronic disease that affect women. To overcome this disease, effective medical treatment is required. Early detection of the disease plays an important role for suitable medication and survival of patient. To identify the breast cancer in the patients, standard imaging technique mammography is used. Due to the subtle and varied nature of cancer tissues interpreting mammogram images can be a challenge to doctors. Machine learning (ML) and Deep Learning (DL) techniques offer promising solutions that provide efficient breast cancer detection from mammograms. In this review paper a comprehensive review of ML and DL algorithms and their applications in mammogram image analysis are presented. Various supervised and unsupervised learning techniques, such as convolutional neural networks (CNNs), support vector machines (SVMs), random forests, and other popular ML and DL models are discussed in paper. The integration of these DL methods that are efficiently used in image preprocessing techniques, feature extraction, and classification strategies. The overall survey focusses on various performance metrics, datasets, and benchmarks used in existing studies. Further the strengths and limitations of different approaches used by various researchers are identified. By understating current research trends this paper aims to contribute to the ongoing development of more accurate and reliable breast cancer detection systems using advanced ML techniques.

Speech emotion recognition (SER) is one of the building blocks of speechenabled personalized applications. However, recognition of emotion from speech is complex due to the variation of emotion intensity (arousal) of the human voice. Data is another challenge for low-resource languages for example, Bangla. To the best of our knowledge, only two datasets are available for SER in Bangla. One of the datasets has not been evaluated using deep learning techniques yet. In this work, we have applied deep learning techniques to available Bangla SER datasets (SUBESCO and BanglaSER) and evaluated their performance. Our major objective is to assess both datasets using identical deep learning methodologies to investigate their usability, a task that has not been undertaken thus far. Convolutional neural networks have been applied to a set of 11 different features of audio data. We have performed extensive experimental study using three different approaches (without cross-validation, cross-validation, and cross-validation with regularization) on three different setups of datasets (combined SUBESCO and Bangla SER, only SUBESCO, and only BanglaSER). Without crossvalidation, more than 70% accuracy has been achieved for all dataset setups. After cross-validation, 68.08% average accuracy and after regularization with cross-validation, 62.62% average accuracy have been achieved on dataset set-up 3 (only BangleSER). The results significantly demonstrate that these datasets are not prepared to develop Bangla SER. Dataset volume is a major challenge for achieving benchmark performance. Hence, the development of a large audio corpus with transcription variation is one of the significant scopes of research in Bangla speech emotion recognition.

Humans have traditionally found it simple to identify emotions from facial expressions, but it is far more difficult for a computer system to do the same. The social signal processing subfield of emotion recognition from facial expression is used in a wide range of contexts, particularly for human-computer interaction. Automatic emotion recognition has been the subject of numerous studies, most of which use a machine learning methodology. The recognition of simple emotions like anger, happiness, contempt, fear, sadness, and surprise, however, continues to be a difficult topic in computer vision. Deep learning has recently drawn increased attention as a solution to a variety of practical issues, including emotion recognition. In this study, we improved the convolutional neural network technique to identify 7 fundamental emotions and evaluated several preprocessing techniques to demonstrate how they affected the CNN performance. This research focuses on improving facial features and expressions based on emotional recognition. By identifying or recognising facial expressions that elicit human responses, it is possible for computers to make more accurate predictions about a person's mental state and to provide more tailored responses. As a result, we examine how a deep learning technique that employs a convolutional neural network might improve the detection of emotions based on facial features (CNN). Multiple facial expressions are included in our dataset, which consists of about 32,298 photos for testing and training. The preprocessing system aids in removing noise from the input image, and the pretraining phase aids in revealing face detection after noise removal, including feature extraction. As a result, the existing paper generates the classification of multiple facial reactions like the seven emotions of the facial acting coding system (FACS) without using the optimization technique, but our proposed paper reveals the same seven emotions of the facial acting coding system.

Artificial intelligence (AI) and machine learning (ML) are employed to make systems smarter. Today, the speech emotion recognition (SER) system evaluates the emotional state of the speaker by investigating his/her speech signal. Emotion recognition is a challenging task for a machine. In addition, making it smarter so that the emotions are efficiently recognized by AI is equally challenging. The speech signal is quite hard to examine using signal processing methods because it consists of different frequencies and features that vary according to emotions, such as anger, fear, sadness, happiness, boredom, disgust, and surprise. Even though different algorithms are being developed for the SER, the success rates are very low according to the languages, the emotions, and the databases. In this paper, we propose a new lightweight effective SER model that has a low computational complexity and a high recognition accuracy. The suggested method uses the convolutional neural network (CNN) approach to learn the deep frequency features by using a plain rectangular filter with a modified pooling strategy that have more discriminative power for the SER. The proposed CNN model was trained on the extracted frequency features from the speech data and was then tested to predict the emotions. The proposed SER model was evaluated over two benchmarks, which included the interactive emotional dyadic motion capture (IEMOCAP) and the berlin emotional speech database (EMO-DB) speech datasets, and it obtained 77.01% and 92.02% recognition results. The experimental results demonstrated that the proposed CNN-based SER system can achieve a better recognition performance than the state-of-the-art SER systems.