Comparative analysis of speech emotion recognition system using MLP, SVM, and CNN algorithms

Emotion recognition plays a pivotal role in advancing Human-Computer Interaction (HCI) by enabling systems to understand and respond to human emotional states. Among various modalities, Speech Emotion Recognition (SER) stands out as a non-invasive, cost-effective, and temporally efficient approach for detecting emotions. This study leverages the RAVDESS dataset, which provides high-quality audio recordings for emotion classification. The proposed methodology involves preprocessing audio signals to remove noise, extracting temporal and spectral features in both time and frequency domains, and implementing machine learning models for multi-class emotion classification. The study evaluates and compares the performance of several classification models, including Random Forest (RF), Multilayer Perceptron (MLP), Support Vector Machine (SVM), Convolutional Neural Networks (CNN), and Decision Trees (DT). Experimental results demonstrate promising accuracy levels, highlighting the potential of these machine learning techniques in SER. This research contributes to the integration of SER within Brain-Computer Interfaces (BCI) and other emotion-aware applications, paving the way for enhanced interactive systems. Keywords – Speech Emotion Recognition -Machine Learning - Random Forest -Multilayer Perceptron - Support Vector Machine - Convolutional Neural Networks - Decision Tree - RAVDESS Dataset

Artificial intelligence (AI) and machine learning (ML) are employed to make systems smarter. Today, the speech emotion recognition (SER) system evaluates the emotional state of the speaker by investigating his/her speech signal. Emotion recognition is a challenging task for a machine. In addition, making it smarter so that the emotions are efficiently recognized by AI is equally challenging. The speech signal is quite hard to examine using signal processing methods because it consists of different frequencies and features that vary according to emotions, such as anger, fear, sadness, happiness, boredom, disgust, and surprise. Even though different algorithms are being developed for the SER, the success rates are very low according to the languages, the emotions, and the databases. In this paper, we propose a new lightweight effective SER model that has a low computational complexity and a high recognition accuracy. The suggested method uses the convolutional neural network (CNN) approach to learn the deep frequency features by using a plain rectangular filter with a modified pooling strategy that have more discriminative power for the SER. The proposed CNN model was trained on the extracted frequency features from the speech data and was then tested to predict the emotions. The proposed SER model was evaluated over two benchmarks, which included the interactive emotional dyadic motion capture (IEMOCAP) and the berlin emotional speech database (EMO-DB) speech datasets, and it obtained 77.01% and 92.02% recognition results. The experimental results demonstrated that the proposed CNN-based SER system can achieve a better recognition performance than the state-of-the-art SER systems.

In the field of medicine, with the introduction of computer systems that can collect and analyze massive amounts of data, many non-invasive diagnostic methods are being developed for a variety of conditions. In this study, our aim is to develop a non-invasive method of classifying respiratory sounds that are recorded by an electronic stethoscope and the audio recording software that uses various machine learning algorithms.In order to store respiratory sounds on a computer, we developed a cost-effective and easy-to-use electronic stethoscope that can be used with any device. Using this device, we recorded 17,930 lung sounds from 1630 subjects.We employed two types of machine learning algorithms; mel frequency cepstral coefficient (MFCC) features in a support vector machine (SVM) and spectrogram images in the convolutional neural network (CNN). Since using MFCC features with a SVM algorithm is a generally accepted classification method for audio, we utilized its results to benchmark the CNN algorithm. We prepared four data sets for each CNN and SVM algorithm to classify respiratory audio: (1) healthy versus pathological classification; (2) rale, rhonchus, and normal sound classification; (3) singular respiratory sound type classification; and (4) audio type classification with all sound types. Accuracy results of the experiments were; (1) CNN 86%, SVM 86%, (2) CNN 76%, SVM 75%, (3) CNN 80%, SVM 80%, and (4) CNN 62%, SVM 62%, respectively.As a result, we found out that spectrogram image classification with CNN algorithm works as well as the SVM algorithm, and given the large amount of data, CNN and SVM machine learning algorithms can accurately classify and pre-diagnose respiratory audio.

Speech is a powerful means of expressing thoughts, emotions, and perspectives. However, accurately determining the emotions conveyed through speech remains a challenging task. Existing manual methods for analyzing speech to recognize emotions are prone to errors, limiting our understanding and response to individuals’ emotional states. To address diverse accents, an automated system capable of real-time emotion prediction from human speech is needed. This paper introduces a speech emotion recognition (SER) system that leverages supervised learning techniques to tackle cross-accent diversity. Distinctively, the system extracts a comprehensive set of nine speech features—Zero Crossing Rate, Mel Spectrum, Pitch, Root Mean Square values, Mel Frequency Cepstral Coefficients, chroma-stft, and three spectral features (Centroid, Contrast, and Roll-off) for refined speech signal processing and recognition. Seven machine learning models are employed, encompassing Random Forest, Logistic Regression, Decision Tree, Support Vector Machines, Gaussian Naive Bayes, K-Nearest Neighbors, ensemble learning, and four individual, hybrid deep learning models including Long short-term memory (LSTM) and 1-Dimensional Convolutional Neural Network (1D-CNN) with stratified cross-validation. Audio samples from diverse English regions are combined to train the models. The performance evaluation results of conventional machine learning and deep learning models indicate that the Random Forest-based feature selection model achieves the highest accuracy of up to 76% among the conventional machine learning models. Simultaneously, the 1D-CNN model with stratified cross-validation reaches up to 99% accuracy. The proposed framework enhances the cross-accent emotion recognition accuracy up to 86.3%, 89.87%, 90.27%, and 84.96% by margins of 14.71%, 10.15%, 9.6%, and 16.52% respectively.

When we are speaking with someone, the first thing we will notice about them is their face. We can easily guess their emotional state by looking at their face, such as which feeling they are experiencing. When we're talking indirectly, however, we can't make the same prediction. As a result, a human may predict the feelings of another human being to some extent. However, a machine such as a computer is incapable of doing so. When we handed up The emotional state of a human being cannot be predicted using an audio clip as input to a computer. As a result, we devised a technique dubbed "Speech Emotion Recognition"[SER] to address this issue. Different strategies can be used to address this voice emotion recognition challenge, but the accuracy is what matters. So we solved this difficulty by comparing the accuracies of two technologies. Here, The most efficient technique for voice emotion detection is the convolution neural network algorithm, which uses several modules for emotion recognition and classifiers to discriminate emotions like happy, surprise, anger, neutral mood, sadness, and so on. [IN-DEE LEARNING] Emotions play a vital role in everyone's lives. Everyone wants to know what other people are up to. As if we can say what the other person thinks when we're talking directly. I'm referring about the facial expressions as well as the tone of voice. However, when it comes to phone calls, we don't know and can't forecast what the person is thinking as an emotion, thus to fix this, we've turned to a new trend known as "Speech Emotion Recognition"[SER]. So, in order to tackle this problem, we used the Machine Learning Language, which includes an efficient algorithm known as SVM, as well as the database Ravdess[Machine Learning].

Separation of pre-recorded messages (Interactive Voice Response, IVR) from live speech fragments in real-time plays a significant role in speech emotion recognition (SER) systems, unwanted calls filtering, automatic detection of answering machine responses, reduction of stored record sizes, voice mail spam filtration, etc. The problem complexity is that, unlike with silent, music, and noise fragments studied by the conventional voice activity recognition (VAD), IVR usually contains speech. Three classifiers for live speech fragments detection in phone call records are considered: based on the support vector machine (SVM), gradient boosting (XGBoost) and convolutional neural network (CNN). The Geneva Minimalistic Acoustic Parameter Set for XGBoost and SVM, and log-spectrograms and gammatonegrams for CNN were used for feature representation of audio fragments. Experiments with a dataset of phone calls demonstrate comparable quality (around 0.96 according to the F1-averaged measure) of the considered algorithms with CNN having a advantage (0.98).

An ensemble 1D-CNN-LSTM-GRU model with data augmentation for speech emotion recognition

Unsupervised feature selection and NMF de-noising for robust Speech Emotion Recognition

To date, most of the human emotion recognition systems are intended to sense the emotions and their dominance individually. This paper discusses a fuzzy model for multilevel affective computing based on the dominance dimensional model of emotions. This model can detect any other possible emotions simultaneously at the time of recognition. One hundred and thirty volunteers from various countries with different cultural backgrounds were selected to record their emotional states. These volunteers have been selected from various races and different geographical locations. Twenty-seven different emotions with their strengths in a scale of 5 were questioned through a survey. Recorded emotions were analyzed with the other possible emotions and their levels of dominance to build the fuzzy model. Then this model was integrated into a fuzzy emotion recognition system using three input devices of mouse, keyboard and the touch screen display. Support vector machine classifier detected the other possible emotions of the users along with the directly sensed emotion. The binary system (non-fuzzy) sensed emotions with an incredible accuracy of 93 %. However, it only could sense limited emotions. By integrating this model, the system was able to detect more possible emotions at a time with slightly lower recognition accuracy of 86 %. The recorded false positive rates of this model for four emotions were measured at 16.7 %. The resulted accuracy and its false positive rate are among the top three accurate human emotion recognition (affective computing) systems.

Mangosteen contains a substance known as Xanthone, a phytochemical compound with the distinctive red component in mangosteen that is known for its properties as an anticancer, antibacterial, and anti-inflammatory agent. Additionally, Xanthone has the potential to strengthen the immune system, promote overall health, support mental well-being, maintain microbial balance in the body, and improve joint flexibility. The mangosteen fruit is consumable when it reaches maturity, displaying a dark purplish-black color. Besides the edible part of the fruit, the peel also possesses remarkable medicinal properties. To detect whether the fruit is ripe or not, this research employs image processing techniques. The study utilizes the HSV (Hue, Saturation, and value) color space method, CNN (Convolutional Neural Network) algorithm, and SVM (Support Vector Machine) algorithm. These methods and algorithms are chosen for their relatively high accuracy levels. The dataset used in this research is obtained from mangosteen datasets available on Kaggle. The results of this study indicate that the HSV method achieved an accuracy of 86.6%, SVM achieved an accuracy of 87%, and CNN achieved an accuracy of 91.25%. From the achieved accuracies, it is evident that the CNN algorithm yields higher accuracy compared to the others.

With the gradual improvement of people's living standards, the production and drinking of all kinds of food is increasing. People's disease rate has increased compared with before, which leads to the increasing number of medical image processing. Traditional technology cannot meet most of the needs of medicine. At present, convolutional neural network (CNN) algorithm using chaotic recursive diagonal model has great advantages in medical image processing and has become an indispensable part of most hospitals. This paper briefly introduces the use of medical science and technology in recent years. The hybrid algorithm of CNN in chaotic recursive diagonal model is mainly used for technical research, and the application of this technology in medical image processing is analysed. The CNN algorithm is optimized by using chaotic recursive diagonal model. The results show that the chaotic recursive diagonal model can improve the structure of traditional neural network and improve the efficiency and accuracy of the original CNN algorithm. Then, the application research and comparison of medical image processing are performed according to CNN algorithm and optimized CNN algorithm. The experimental results show that the CNN algorithm optimized by chaotic recursive diagonal model can help medical image automatic processing and patient condition analysis.

Speech Emotion Recognition (SER) plays a vital role in various real-world applications, from mental health diagnostics to human-computer interaction. This research examines the integration of Stacked Long Short-Term Memory (LSTM) networks and Hidden Markov Models (HMM) for SER, addressing the limitations of traditional models like Gaussian Mixture Models (GMM) and Support Vector Machines (SVM). While GMMs and SVMs offer decent performance, their inability to model temporal dynamics limits their accuracy in detecting complex emotions. The proposed hybrid model leverages LSTM’s strength in handling sequential data and HMM’s ability to model emotional transitions, making it highly suitable for real-world noisy environments. Preprocessing techniques such as MFCC and LPCC are applied to enhance feature extraction, and benchmark datasets like IEMOCAP and RAVDESS are used for evaluation. Finally, The Systematic review highlights the superior role of the hybrid model's performance on SER and sets the stage for a significant shift in future research in addressing bias and fairness in SER systems by combining LSTM and HMM as hybrid model. In recent studies, context-aware emotion recognition models are commonly evaluated using existing datasets such as IEMOCAP, MELD, DailyDialog, and SEMAINE, which provide conversational and multimodal emotional data. Features used often include acoustic features as MFCCs, spectrograms, prosodic features, linguistic features as BERT or transformer embeddings, and visual cues such as facial landmarks or video frames. Many models also incorporate contextual embeddings from transformers, graph attention networks, or sequential memory modules to capture speaker history and emotional shifts.

Speech Emotion Recognition (SER) allows systems to interpret emotions in human speech, creating more natural and responsive interactions between people and machines. Due to the complex nature of emotion detection, several deep learning techniques have been utilized, yet limited research have focused on optimizing key hyperparameters of Convolutional Neural Network (CNN) for a more efficient system. Hence, this research optimized CNN with Mantis Search Algorithm (MSA) due to its ease of implementation, ability to preserve population diversity during the optimization process, ability to escape from the local optima and balance between exploration and exploitation operators. Audio data for four emotions: anger, fear, happiness and neutrality were acquired from Toronto Emotional Speech Set (TESS) available on Kaggle.com. The audio data were then converted into text using speech-to-text code and preprocessed using Natural Language Processing (NLP) techniques: tokenization, removal of stop words, lemmatization, removal of punctuations and lowercase conversion. Mantis Search Algorithm was then applied to optimize CNN for optimal selection of filter size and learning rate. The optimized CNN (MSA-CNN) was implemented using MATLAB R2023a software. The performance of the system was evaluated and compared with CNN classifier using False Positive Rate (FPR), Specificity (Spec), Sensitivity (Sen), Precision (Prec), Accuracy (Acc), and Recognition Time (RT). The optimized speech emotion recognition system showed improved values over CNN on all the metrics considered.

Speech is the fastest and most natural method for human to communicate. This has led several researches to be done in the field of the interaction effects between human and machine. Hence, it is necessary to design machines which can intelligently recognize the emotion of a human voice. However, we are still far from having a natural interaction between the human and machine because machines cannot distinguish the emotion of the speaker. This has established a new field in the literature, namely the speech emotion recognition systems. The accuracy of these systems depends on various factors such as the number and type of the emotion manners as well as the feature selection and the classifier sort. In this paper, classification methods of the Neural Network (NN), Support Vector Machine (SVM), the combination of NN and SVM (NN-SVM), NN and SVM (NN-SVM), NN and C5.0 (NN-C5.0), C5.0 and SVM (SVM-C5.0), and finally the combination of NN, SVM, and C5.0 (NN-SVM-C5.0) have been verified, and their efficiencies in speech emotion recognition have been compared. The utilized features in this research include energy, power, Zero Crossing Rate (ZCR), pitch, and Mel-scale Frequency Cepstral Coefficients (MFCC). The presented results in this paper demonstrate that using the proposed NN-C5.0 classification method is more efficient in recognizing the emotion states-to the extent of 6%- to 30% depending on the number of emotions states-than SVM, NN, and other aforementioned combinations of classification methods.

This study presents a comparative analysis of the classification performance of facial and emotion recognition systems using Machine Learning (ML) and Deep Learning (DL) algorithms. The primary objective of this work is to evaluate the applicability of emotion recognition in fields such as psychotherapy and crime analysis, using the FER-2013 dataset.The study was conducted with ML algorithms such as Support Vector Machines (SVM), Random Forest (RF), K-Nearest Neighbor (KNN), Decision Trees, and Gradient Boosting, as well as the DL algorithm, Convolutional Neural Network (CNN). Supported by the application of feature selection, preprocessing, and feature extraction techniques, the model’s performance was measured using standard metrics such as accuracy, precision, F1-score, and AUC-ROC.The experimental results showed that the highest classification accuracy (46.61%) was achieved with the CNN model. While the ML models generally offered lower accuracy, they provided advantages in terms of computational efficiency in specific scenarios.This study aims to contribute to the literature by providing a comparative analysis of ML and DL algorithms and by highlighting the effect of data preprocessing on performance. The findings set targets for future work, such as real-time system integration and hybrid model development.