Learning affective representations based on magnitude and dynamic relative phase information for speech emotion recognition

Features for speech emotion recognition are usually dominated by the spectral magnitude information while they ignore the use of the phase spectrum because of the difficulty of properly interpreting it. Motivated by recent successes of phase-based features for speech processing, this paper investigates the effectiveness of phase information for whispered speech emotion recognition. We select two types of phase-based features (i.e., modified group delay features and all-pole group delay features), both which have shown wide applicability to all sorts of different speech analysis and are now studied in whispered speech emotion recognition. When exploiting these features, we propose a new speech emotion recognition framework, employing outer product in combination with power and L2 normalization. The according technique encodes any variable length sequence of the phase-based features into a fixed dimension vector regardless of the length of the input sequence. The resulting representation is fed to train a classification model with a linear kernel classifier. Experimental results on the Geneva Whispered Emotion Corpus database, including normal and whispered phonation, demonstrate the effectiveness of the proposed method when compared with other modern systems. It is also shown that, combining phase information with magnitude information could significantly improve performance over the common systems solely adopting magnitude information.

Conventional Speech Emotion Recognition (SER) approaches put more emphasis on extracting magnitude spectrum-based features, such as Mel Frequency Cepstral Coefficients (MFCCs), and Mel spectrogram. However, phase information is ignored due to signal processing difficulties such as the phase wrapping issue. This work develops a multichannel Convolution Neural Network-Bidirectional Long Short Term Memory (CNN-BLSTM) architectures with an attention mechanism for speaker-independent SER by considering phase and magnitude spectrum-based features. The phase-based features are extracted using the Modified Group Delay Function (MODGD). The obtained phase features are combined with MFCC features. The CNN-BLSTM network extract learned representation from magnitude and phase features. The learned representation from MFCCs and MODGD are combined and given as an input to the Support Vector Machine (SVM) for classification. The Deep Canonical Correlation Analysis (DCCA) is used to maximize the correlation between magnitude and phase information to improve the conventional SER system’s performance. Here the IEMOCAP database is used for performance analysis. The experimental results show improvement over MFCC features and existing approaches for unimodal SER. In this work, we also developed real-time Web server application for the proposed architecture.

A new method of presenting magnetic resonance phase information is described and an example of its application given in the context of velocity phase-encoded MRI of blood flow in the aorta. The method takes as its starting point Zernike's technique of phase contrast microscopy. It exploits the parallel between the transform plane in Fourier optics and kappa-space in MRI. In the example described two datasets are acquired, one with and the other without velocity encoding, as in conventional phase-encoded velocity imaging. A dataset is formed which is corrected for unwanted phase variations caused by static field inhomogeneity. The method then effectively combines phase and magnitude information into a single image. The technique is complementary to existing methods of displaying phase information.

In this paper, a new model extraction method for band-pass filters with convolutional neural network (CNN) based on magnitude information only is introduced. A data augmentation approach and a data decentralization process to improve the CNN model performance by statistical analysis are proposed. A correction process utilizing real test data is introduced to improve the model performance. Compared to the existing model extraction method, this CNN model approach needs no phase information that is highly dependent on the loading circuit of a filter. Merely using the magnitude information, a good fitness for the response of high-order filters with complicated topology is achieved with this method. CNN model demonstrates a faster convergence compared to the traditional fully connected neural model.

The acoustic cues that convey emotion in speech are similar to those that convey emotion in music, and recognition of emotion in both of these types of cue recruits overlapping networks in the brain. Given the similarities between music and speech prosody, developmental research is uniquely positioned to determine whether recognition of these cues develops in parallel. In the present study, we asked 60 children aged 6 to 11 years, and 51 university students, to judge the emotions of 10 musical excerpts, 10 inflected speech clips, and 10 affect burst clips. We presented stimuli intended to convey happiness, sadness, anger, fear, and pride. Each emotion was presented twice per type of stimulus. We found that recognition of emotions in music and speech developed in parallel, and adult-levels of recognition develop later for these stimuli than for affect bursts. We also found that sad stimuli were most easily recognised, followed by happiness, fear, and then anger. In addition, we found that recognition of emotion in speech and affect bursts can predict emotion recognition in music stimuli independently of age and musical training. Finally, although proud speech and affect bursts were not well recognised, children aged eight years and older showed adult-like responses in recognition of proud music.

Combining evidences from magnitude and phase information using VTEO for person recognition using humming

Morphological filters are investigated and employed for detecting and visualizing objects within an image. The techniques developed here will be employed on NASA's Earth Observing System (EOS) satellite data products for the purpose of anomaly detection. Previous efforts have shown the phase information in the spectral domain to be more significant than the magnitude information in representing the location of objects in an image. The magnitude information does provide some useful information for object location, but it is also sensitive to image illumination, blurring, and magnification variations, all of which influence the performance of object detection algorithms. Magnitude reduction techniques in the spectral domain can dramatically improve subsequent object detection methods by causing them to rely less on the magnitude and more on the phase information of the image. We propose three new improvements to our object enhancement and detection techniques. Our first method is an enhancement to our previous magnitude-reduction technique. Our second improvement is a modification of our Rational Morphological Filters in which we raise our resulting image to a power, thereby magnifying our feature detection capability. Third, we look at speed enhancement by utilizing Hartley and Walsh Transforms in place of classical Fourier techniques.

In the recent years, many research works have been published using speech related features for speech emotion recognition, however, recent studies show that there is a strong correlation between emotional states and glottal features. In this work, Mel-frequency cepstralcoefficients (MFCCs), linear predictive cepstral coefficients (LPCCs), perceptual linear predictive (PLP) features, gammatone filter outputs, timbral texture features, stationary wavelet transform based timbral texture features and relative wavelet packet energy and entropy features were extracted from the emotional speech (ES) signals and its glottal waveforms(GW). Particle swarm optimization based clustering (PSOC) and wrapper based particle swarm optimization (WPSO) were proposed to enhance the discerning ability of the features and to select the discriminating features respectively. Three different emotional speech databases were utilized to gauge the proposed method. Extreme learning machine (ELM) was employed to classify the different types of emotions. Different experiments were conducted and the results show that the proposed method significantly improves the speech emotion recognition performance compared to previous works published in the literature.

Speech emotion recognition is a still challenging problem despite having been investigated over the last couple of decades. Conventional speech emotion recognition performance is low, but this may be improved by considering new features and an annotation method. In this paper, firstly we use glottal features for speech emotion recognition to improve its performance because the emotions are related to glottal characteristics. Secondly emotional labels used as “correct” emotions are usually annotated by multiple annotators in the conventional approach, but this may be ambiguous due to the confusion created by using each annotator's criterion. In this paper, a single annotator gives emotional labels in an aim to obtain a unified criterion and we use these labels for speech emotion recognition to improve its performance. Finally, context information which is included in a speech segment before the utterance being annotated, will be considered.

For certain types of signals, perfect reconstruction is not essential. It is sufficient to have knowledge of either the magnitude or the phase spectrum only. Filter banks which introduce phase distortion are used for encoding speech signals. These schemes have an inherent redundancy; i.e., the samples are still transmitted at the sampling rate. It is shown how magnitude or phase information can be encoded at only half the sampling rate in a filter bank formalism. >

-- Speech signals are being considered as most effective means of communication between human beings. Many researchers have found different methods or systems to identify emotions from speech signals. Here, the various features of speech are used to classify emotions. Features like pitch, tone, intensity are essential for classification. Large number of the datasets are available for speech emotion recognition. Firstly, the extraction of features from speech emotion is carried out and then another important part is classification of emotions based upon speech. Hence, different classifiers are used to classify emotions such as Happy, Sad, Anger, Surprise, Neutral, etc. Although, there are other approaches based on machine learning algorithms for identifying emotions. Speech Emotion Recognition is a current research topic because of its wide range of applications and it became a challenge in the field of speech processing too. We have carried out a brief study on Speech Emotion Analysis along with Emotion Recognition. Speech Emotion Recognition (SER) can be defined as extraction of the emotional state of the speaker from his or her speech signal. There are few universal emotions including Neutral, Anger, we have worked on different tools to be used in SER. SER is tough because emotions are subjective and annotating audio is challenging task. Emotion recognition is the part of speech recognition which is gaining more popularity and need for it increases enormously. We have classified based on different type of emotions to detect from speech. Key Words: Speech Emotion Recognition, Affective Computing, Machine Learning, Deep Learning, Audio Signal Processing, Emotion Classification, Feature Extraction, Prosodic Features, Spectral Features, Mel-Frequency Cepstral Coefficients (MFCCs), Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), Attention Mechanisms, Multimodal Emotion Recognition, Speaker-Independent SER, Real-Time Emotion Detection, Noise-Robust Emotion Recognition, Data Augmentation, Emotion-Aware Applications

Iris pattern matching is a key step in iris patter recognition system. This paper proposes a new approach for iris pattern matching using block-based spectral angle measure (SAM). The iris patterns in this paper are extracted using Log-Gabor wavelet method. The proposed SAM method will capture not only the phase information, but also the magnitude information. The block based method will enable the flexibility in the pattern matching step, which improves the accuracy and is more tolerable of the pattern dilation/constrain resulted from imperfect preprocessing. The preliminary experimental results show that the proposed method has an encouraging performance.

Accurate blood vessel segmentation plays a crucial role in non-invasive blood flow velocity measurement based on complex-valued magnetic resonance images. We propose a specific snake active contour model-based blood vessel segmentation framework for complex-valued magnetic resonance images. The proposed framework combines both magnitude and phase information from a complex-valued image representation to obtain an optimum segmentation result. Magnitude information of the complex-valued image provides a structural localization of the target object, while phase information identifies the existence of flowing matters within the object. Snake active contour model, which models the segmentation procedure as a force-balancing physical system, is being adopted as a framework for this work due to its interactive, dynamic, and customizable characteristics. Two snake-based segmentation models are developed to produce a more accurate segmentation result, namely the Model-constrained Gradient Vector Flow-snake (MC GVF-snake) and Stochastic-snake. MC GVF-snake elaborates a prior knowledge on common physical structure of the target object to restrict and guide the segmentation mechanism, while Stochastic-snake implements the simulated annealing stochastic procedure to produce improved segmentation accuracy. The developed segmentation framework has been evaluated on actual complex-valued MRI images, both in noise-free and noisy simulated conditions. Evaluation results indicate that both of the developed algorithms give an improved segmentation performance as well as increased robustness, in comparison to the conventional snake algorithm.

Dynamic texture is a spatially repetitive, time-varying visual pattern that forms an image sequence with some spatio-temporal stationary properties. This paper proposes a dynamic texture classification algorithm based on the magnitude information and the phase information in dual-tree complex wavelet transform domain. The variance and entropy of the complex wavelet coefficients are used to form the dynamic texture feature vector. Experimental results show that the algorithm has a good effect and performance on dynamic texture classification.

This paper develops a novel face recognition technique called Complete Gabor Fisher Classifier (CGFC). Different from existing techniques that use Gabor filters for deriving the Gabor face representation, the proposed approach does not rely solely on Gabor magnitude information but effectively uses features computed based on Gabor phase information as well. It represents one of the few successful attempts found in the literature of combining Gabor magnitude and phase information for robust face recognition. The novelty of the proposed CGFC technique comes from (1) the introduction of a Gabor phase-based face representation and (2) the combination of the recognition technique using the proposed representation with classical Gabor magnitude-based methods into a unified framework. The proposed face recognition framework is assessed in a series of face verification and identification experiments performed on the XM2VTS, Extended YaleB, FERET, and AR databases. The results of the assessment suggest that the proposed technique clearly outperforms state-of-the-art face recognition techniques from the literature and that its performance is almost unaffected by the presence of partial occlusions of the facial area, changes in facial expression, or severe illumination changes. 1.