Investigating Graph-based Features for Speech Emotion Recognition

Emotion is a natural feeling which is distinguished from reasoning or knowledge, it is a strong feeling derived from one’s circumstance or surroundings. With the increase in man to machine interaction, speech analysis has become an integral part in reducing the gap between physical and digital world. An important sub field within this domain is the recognition of emotion in speech signals, which was traditionally studied in linguistics and psychology. Speech emotion recognition is a field having diverse applications. When implemented the Speech Emotion Recognition (SER) will be able to understand different human emotion such as anger, fear, happiness, sadness etc. Speech is a medium of expression of one’s perspective or feelings to other. Emotion recognition from audio signal requires feature extraction and classifier training. The feature vector consists of elements of the audio signal which characterize speaker specific features such as tone, pitch, energy, which is crucial to train the classifier model to recognize a particular emotion accurately. Thus, with the help of SER we can make conversations between human and computer more realistic and natural. Automatic Speech Emotion Recognition (SER) is a current research topic in the field of Human Computer Interaction (HCI) with wide range of applications. The speech features such as, Mel Frequency cepstrum coefficients (MFCC) and Mel Energy Spectrum Dynamic Coefficients (MEDC) are extracted from speech utterance. The Support Vector Machine (SVM) is used as classifier to classify different emotional states such as anger, happiness, sadness, neutral, fear, from Berlin emotional database.

In the last decade, there has been significant research into Automatic Speech Emotion Recognition (SER). The primary goal of SER is to improve human-machine interfaces. It can also monitor someone's psychological state for lie detection applications. Recently, speech emotion recognition has found uses in medicine and forensics. This paper recognizes 7 emotions using pitch and prosody features. The majority of speech features used here are in the time domain. A Support Vector Machine (SVM) classifier categorizes the emotions. The Berlin emotional database was used for this task. A good recognition rate of 81% was achieved. The reference paper for this work recognized 4 emotions and obtained a 94.2% recognition rate. However, the reference paper used a more complex hybrid classifier, while this work focuses on recognizing more emotions with a simpler model. . KeyWords: Blindness, Smart Stick, BlinDar, GPS, ESP8266, Internet of Things, RF Tx/Rx, MQ2.

This paper presents an ultra wideband (UWB) channel sounding scheme with a technique for time of arrival (TOA) and angle of arrival (AOA) estimation using transformation between time and frequency domain signals. Since the measured UWB power spectrum can be transformed into a time domain signal, both frequency and time domain signals are available for the channel estimation. We propose the TOA estimation using a peak search over time domain signal, and the AOA estimation using MUSIC and AOA histograms. The AOA histogram is counted for AOA samples of discrete frequencies over the UWB bandwidth. Comparing both AOA histograms before and after TOA subtraction, we can determine pairing the elements of TOA vector and those of AOA vector. The anechoic chamber tests indicated that both TOAs and AOAs were precisely resolved for two-path model, even if either TOAs or AOAs were too close to resolve.

Features for speech emotion recognition are usually dominated by the spectral magnitude information while they ignore the use of the phase spectrum because of the difficulty of properly interpreting it. Motivated by recent successes of phase-based features for speech processing, this paper investigates the effectiveness of phase information for whispered speech emotion recognition. We select two types of phase-based features (i.e., modified group delay features and all-pole group delay features), both which have shown wide applicability to all sorts of different speech analysis and are now studied in whispered speech emotion recognition. When exploiting these features, we propose a new speech emotion recognition framework, employing outer product in combination with power and L2 normalization. The according technique encodes any variable length sequence of the phase-based features into a fixed dimension vector regardless of the length of the input sequence. The resulting representation is fed to train a classification model with a linear kernel classifier. Experimental results on the Geneva Whispered Emotion Corpus database, including normal and whispered phonation, demonstrate the effectiveness of the proposed method when compared with other modern systems. It is also shown that, combining phase information with magnitude information could significantly improve performance over the common systems solely adopting magnitude information.

Learning affective representations based on magnitude and dynamic relative phase information for speech emotion recognition

Speech emotion recognition is a still challenging problem despite having been investigated over the last couple of decades. Conventional speech emotion recognition performance is low, but this may be improved by considering new features and an annotation method. In this paper, firstly we use glottal features for speech emotion recognition to improve its performance because the emotions are related to glottal characteristics. Secondly emotional labels used as “correct” emotions are usually annotated by multiple annotators in the conventional approach, but this may be ambiguous due to the confusion created by using each annotator's criterion. In this paper, a single annotator gives emotional labels in an aim to obtain a unified criterion and we use these labels for speech emotion recognition to improve its performance. Finally, context information which is included in a speech segment before the utterance being annotated, will be considered.

In the recent years, many research works have been published using speech related features for speech emotion recognition, however, recent studies show that there is a strong correlation between emotional states and glottal features. In this work, Mel-frequency cepstralcoefficients (MFCCs), linear predictive cepstral coefficients (LPCCs), perceptual linear predictive (PLP) features, gammatone filter outputs, timbral texture features, stationary wavelet transform based timbral texture features and relative wavelet packet energy and entropy features were extracted from the emotional speech (ES) signals and its glottal waveforms(GW). Particle swarm optimization based clustering (PSOC) and wrapper based particle swarm optimization (WPSO) were proposed to enhance the discerning ability of the features and to select the discriminating features respectively. Three different emotional speech databases were utilized to gauge the proposed method. Extreme learning machine (ELM) was employed to classify the different types of emotions. Different experiments were conducted and the results show that the proposed method significantly improves the speech emotion recognition performance compared to previous works published in the literature.

Introduction à la neuropsychologie des épilepsies

Emotion recognition plays a pivotal role in advancing Human-Computer Interaction (HCI) by enabling systems to understand and respond to human emotional states. Among various modalities, Speech Emotion Recognition (SER) stands out as a non-invasive, cost-effective, and temporally efficient approach for detecting emotions. This study leverages the RAVDESS dataset, which provides high-quality audio recordings for emotion classification. The proposed methodology involves preprocessing audio signals to remove noise, extracting temporal and spectral features in both time and frequency domains, and implementing machine learning models for multi-class emotion classification. The study evaluates and compares the performance of several classification models, including Random Forest (RF), Multilayer Perceptron (MLP), Support Vector Machine (SVM), Convolutional Neural Networks (CNN), and Decision Trees (DT). Experimental results demonstrate promising accuracy levels, highlighting the potential of these machine learning techniques in SER. This research contributes to the integration of SER within Brain-Computer Interfaces (BCI) and other emotion-aware applications, paving the way for enhanced interactive systems. Keywords – Speech Emotion Recognition -Machine Learning - Random Forest -Multilayer Perceptron - Support Vector Machine - Convolutional Neural Networks - Decision Tree - RAVDESS Dataset

In this paper, we present an approach that using articulatory features (AFs) derived from spectral features for speech emotion recognition. Also, we investigated the combination of AFs and spectral features. Systems based on AFs only and combined spectral-articulatory features are tested on the CASIA Mandarin emotional corpus. Experiments results show that AFs alone are not suitable for speech emotion recognition and that the combination of spectral features and AFs don’t improve the performance of the system that using only spectral features.

The Dual-Tree Complex Wavelet Packet Transform (DT-CWPT) has been successfully implemented in numerous field because it introduces limited redundancy, provides approximately shift-invariance and geometrically oriented signal in multiple dimensions where these properties are lacking in traditional wavelet transform. This paper investigates the performance of features extracted using DT-CWPT algorithms which are quantified using k-Nearest Neighbors (k-NN) and Support Vector Machine (SVM) classifiers for detecting voice pathologies. Decomposition is done on the voice signals using Shannon and Approximate entropy (ApEn) to signify the complexity of voice signals in time and frequency domain. Feature selection methods using the ReliefF algorithm and Genetic algorithm (GA) are applied to obtain the optimum features for multiclass classification. It is observed that the best accuracies obtained using DT-CWPT with ApEn entropy are 91.15 % for k-NN and 93.90 % for SVM classifiers. The proposed work provides a promising detection rate for multiple voice disorders and is useful for the development of computer-based diagnostic tools for voice pathology screening in health care facilities.

In automatic speech and speech emotion recognition, a good quality of input speech signal is often required. The hit rate of recognizers is lowered by degradation of speech quality due to noise. Blind source separation can be used to enhance the speech signal as a part of preprocessing techniques. This paper presents a multi channel linear blind source separation method that can be applied even in underdetermined case i.e. when the number of source signals is higher than the number of sensors. Experiments have shown that our system outperforms conventional time-frequency binary masking in both determined and underdetermined cases and significantly increases the hit rate of speech recognizers.

Speech signal feature extraction is an important part of speech recognition system. We present Tucker decomposition to extract speech feature. Firstly, the preprocessed speech signal is decomposed via three-level Wavelet transform, and the information in different scales is obtained. Next, the conventional feature parameters are extracted from the different scales, and a 3-order speech tensor (frames, scales, feature parameters) could be created. Then, the tensor is decomposed by Tucker decomposition, and projection matrices in different mode are obtained. Thirdly, matrix product is performed between speech tensor and projection matrices in each mode, and mapped results are metricized. Finally, feature system in high order space is built, in other words, speech feature matrices are obtained. The feature system can fully express speech signal features. These matrices can be used for model training and speech recognition. Numerical experiments support the advantage of Tucker decomposition over conventional methods for speech signal feature extraction, furthermore, it is robust to noisy speech.

In the present work we outline the first computer game which is operated by real-time emotion recognition from speech. Urgency is detected from the voice of a player and conveyed to control parameters like speed during a race or accuracy while shooting on a target for a biathlon sports simulation. Moreover, the game showcases the world's first automatic speech emotion recognition which is based on the Component Process Model, a fundamental theory of cognitive psychology.

Emotion recognition from speech is essential for human–computer interaction (HCI) and affective computing, with applications in virtual assistants, healthcare, and education. Although deep learning has made significant advancements in Automatic Speech Emotion Recognition (ASER), the challenge still exists in the task given variation in speakers, subtle emotional expressions, and environmental noise. Practical deployment in this context depends on a strong, fast, scalable recognition system. This work introduces a new framework combining DenseNet121, especially fine-tuned for the crowd-sourced emotional multimodal actors dataset (CREMA-D), with the convolutional block attention module (CBAM). While DenseNet121’s effective feature propagation captures rich, hierarchical patterns in the speech data, CBAM improves the focus of the model on emotionally significant elements by applying both spatial and channel-wise attention. Furthermore, enhancing the input spectrograms and strengthening resistance against environmental noise is an advanced preprocessing pipeline including log-Mel spectrogram transformation and normalization. The proposed model demonstrates superior performance. To make sure the evaluation is strong even if there is a class imbalance, we point out important metrics like an Unweighted Average Recall (UAR) of 71.01% and an F1 score of 71.25%. The model also gets a test accuracy of 71.26% and a precision of 71.30%. These results establish the model as a promising solution for real-world speech emotion detection, highlighting its strong generalization capabilities, computational efficiency, and focus on emotion-specific features compared to recent work. The improvements demonstrate practical flexibility, enabling the integration of established image recognition techniques and allowing for substantial adaptability in various application contexts.