Deep learning and SVM‐based emotion recognition from Chinese speech for smart affective services

With the rapid development of artificial intelligence technology of deep learning method, it has been applied to many fields, especially to life science. In this paper, a novel approach for the task of voiceprint recognition was proposed. The combination of deep belief network (DBN) and support vector machine (SVM) was used to identify the voiceprint of 10 different individuals. Based on a 24-dimension Mel Frequency Cepstrum Coefficient (MFCC), the authors extracted 256-dimension deep voiceprint features via DBN model that is developed by stacking three layers of Restricted Boltzmann Machine (RBM), and conducted voiceprint recognition by approach of SVM. According to the recognition results, the new approach can significantly improve both of accuracy and efficiency when it was compared with traditional voiceprint features and recognition models. The low dimensional features of voiceprint was extracted into higher dimensions by DBN model, while SVM can avoid the elevation of computation complexity caused by the increases of feature dimension. The combined strengths have been fully expressed by the experimental results.

Emotion recognition from single-channel EEG signals using a two-stage correlation and instantaneous frequency-based filtering method

Individuals with cochlear implants (CIs) show reduced word and auditory emotion recognition abilities relative to their peers with normal hearing. Modern CI processing strategies are designed to preserve acoustic cues requisite for word recognition rather than those cues required for accessing other signal information (e.g., talker gender or emotional state). While word recognition is undoubtedly important for communication, the inaccessibility of this additional signal information in speech may lead to negative social experiences and outcomes for individuals with hearing loss. This study aimed to evaluate whether the emphasis on word recognition preservation in CI processing has unintended consequences on the perception of other talker information, such as emotional state. Twenty-four young adult listeners with normal hearing listened to sentences and either reported a target word in each sentence (word recognition task) or selected the emotion of the talker (emotion recognition task) from a list of options (Angry, Calm, Happy, and Sad). Sentences were blocked by task type (emotion recognition versus word recognition) and processing condition (unprocessed versus 8-channel noise vocoder) and presented randomly within the block at three signal-to-noise ratios (SNRs) in a background of speech-shaped noise. Confusion matrices showed the number of errors in emotion recognition by listeners. Listeners demonstrated better emotion recognition performance than word recognition performance at the same SNR. Unprocessed speech resulted in higher recognition rates than vocoded stimuli. Recognition performance (for both words and emotions) decreased with worsening SNR. Vocoding speech resulted in a greater negative impact on emotion recognition than it did for word recognition. These data confirm prior work that suggests that in background noise, emotional prosodic information in speech is easier to recognize than word information, even after simulated CI processing. However, emotion recognition may be more negatively impacted by background noise and CI processing than word recognition. Future work could explore CI processing strategies that better encode prosodic information and investigate this effect in individuals with CIs as opposed to vocoded simulation. This study emphasized the need for clinicians to consider not only word recognition but also other aspects of speech that are critical to successful social communication.

Unsupervised feature selection and NMF de-noising for robust Speech Emotion Recognition

The speech signal consists of linguistic information and also paralinguistic one such as emotion. The modern automatic speech recognition systems have achieved high performance in neutral style speech recognition, but they cannot maintain their high recognition rate for spontaneous speech. So, emotion recognition is an important step toward emotional speech recognition. The accuracy of an emotion recognition system is dependent on different factors such as the type and number of emotional states and selected features, and also the type of classifier. In this paper, a modular neural-support vector machine (SVM) classifier is proposed, and its performance in emotion recognition is compared to Gaussian mixture model, multi-layer perceptron neural network, and C5.0-based classifiers. The most efficient features are also selected by using the analysis of variations method. It is noted that the proposed modular scheme is achieved through a comparative study of different features and characteristics of an individual emotional state with the aim of improving the recognition performance. Empirical results show that even by discarding 22% of features, the average emotion recognition accuracy can be improved by 2.2%. Also, the proposed modular neural-SVM classifier improves the recognition accuracy at least by 8% as compared to the simulated monolithic classifiers.

Emotions are quite important in our daily communications and recent years have witnessed a lot of research works to develop reliable emotion recognition systems based on various types data sources such as audio and video. Since there is no apparently visual information of human faces, emotion analysis based on only audio data is a very challenging task. In this work, a novel emotion recognition is proposed based on robust features and machine learning from audio speech. For a person independent emotion recognition system, audio data is used as input to the system from which, Mel Frequency Cepstrum Coefficients (MFCC) are calculated as features. The MFCC features are then followed by discriminant analysis to minimize the inner-class scatterings while maximizing the inter-class scatterings. The robust discriminant features are then applied with an efficient and fast deep learning approach Neural Structured Learning (NSL) for emotion training and recognition. The proposed approach of combining MFCC, discriminant analysis and NSL generated superior recognition rates compared to other traditional approaches such as MFCC-DBN, MFCC-CNN, and MFCC-RNN during the experiments on an emotion dataset of audio speeches. The system can be adopted in smart environments such as homes or clinics to provide affective healthcare. Since NSL is fast and easy to implement, it can be tried on edge devices with limited datasets collected from edge sensors. Hence, we can push the decision-making step towards where data resides rather than conventionally processing of data and making decisions from far away of the data sources. The proposed approach can be applied in different practical applications such as understanding peoples’ emotions in their daily life and stress from the voice of the pilots or air traffic controllers in air traffic management systems.

Precision mechanism is widely used for various industry applications. Quality inspection for precision mechanism is essential for manufacturers to assure the product leaving factory with expected quality. In this paper, we propose a novel automated fault detection method, named Tilear, based on a Deep Belief Network (DBN) auto-encoder. DBN is a probabilistic generative model, composed by stacked Restricted Boltzmann Machines. With its RBM-layer-wise training methods, DBN can perform fast inference and extract high level feature of the inputs. By unfolding the stacked RBMs symmetrically, a DBN auto-encoder is constructed to reconstruct the inputs as closely as possible. Based on the DBN auto-encoder, Tilear is structured in two parts: training and decision-making. During training, Tilear is trained with the signals only from good samples, which enables the trained DBN auto-encoder only know how to reconstruct signals of good samples. In the decision-making part, comparing the recorded signal from test sample and the Tilear reconstructed signal, allows to measure how well a recording from a test sample matches the DBN auto-encoder model learned from good samples. A reliable decision could be made. We perform experiments on two different precision mechanisms: precision electromotors and greasing control units. The feasibility of Tilear was demonstrated first. Additionally, performance of Tilear on the acquired electromotor dataset was compared with the state-of-the-art machine learning based fault detection technique, support vector machine (SVM). First result indicates that Tilear excels the SVM in terms of the Area Under the Curve (AUC) obtained from the Receiver Operating Characteristics (ROC) curve plot: 0.960 achieved by Tilear, while 0.941 by SVM.

Emotion recognition aims at determining the state of emotion which is included in the speech or mimics of a person. Emotion recognition from speech is an area related to signal processing and psychology. Acoustic parameters obtained from speech signals through acoustic analysis, which is one of objective evaluation methods, is intensively used in emotion recognition studies. In this paper success in emotion recognition is examined in categorical aspects and the impact of dimensional model on independent emotion recognition success is investigated. Acoustic parameters were subjected to classification with Support Vector Machine in order to determine emotion recognition success. According to the obtained findings, emotion recognition success in categorical structure, dimensional structure and categorical-dimensional structure were 69.5%, 73.3% and 87.1%, respectively. Even if dimensional structure is higher in arousal than in valence, when emotion recognition success is examined on each emotion dimension, valence provided higher success.

In order to effectively improve the expressiveness and classification accuracy of fault signal characteristics of equipment, this paper proposed a fault diagnosis method based on Mel-Frequency Cepstrum Coefficient (MFCC) fusion and Support Vector Machines (SVM). First of all, the MFCC features, the Wavelet Packet Decomposition Energy features and the Zero-crossing rate (ZCR) features of the signal are separately extracted. Then, linearly combining the three features based on the MFCC features to obtain the MFCC fusion features. And the SVM classifier is used to classify the faults. Experiments show that compared with the traditional single MFCC features and Wavelet Packet Decomposition Energy features, the MFCC fusion features and SVM method have higher classification accuracy under the same noise environment.

Research in emotion recognition seeks to develop insights into the temporal properties of emotion. However, automatic emotion recognition from spontaneous speech is challenging due to non-ideal recording conditions and highly ambiguous ground truth labels. Further, emotion recognition systems typically work with noisy high-dimensional data, rendering it difficult to find representative features and train an effective classifier. We tackle this problem by using Deep Belief Networks, which can model complex and non-linear high-level relationships between low-level features. We propose and evaluate a suite of hybrid classifiers based on Hidden Markov Models and Deep Belief Networks. We achieve state-of-the-art results on FAU Aibo, a benchmark dataset in emotion recognition [1]. Our work provides insights into important similarities and differences between speech and emotion.

PurposeUp to date development in sentiment analysis has resulted in a symbolic growth in the volume of study, especially on more subjective text types, namely, product or movie reviews. The key difference between these texts with news articles is that their target is defined and unique across the text. Hence, the reviews on newspaper articles can deal with three subtasks: correctly spotting the target, splitting the good and bad content from the reviews on the concerned target and evaluating different opinions provided in a detailed manner. On defining these tasks, this paper aims to implement a new sentiment analysis model for article reviews from the newspaper.Design/methodology/approachHere, tweets from various newspaper articles are taken and the sentiment analysis process is done with pre-processing, semantic word extraction, feature extraction and classification. Initially, the pre-processing phase is performed, in which different steps such as stop word removal, stemming, blank space removal are carried out and it results in producing the keywords that speak about positive, negative or neutral. Further, semantic words (similar) are extracted from the available dictionary by matching the keywords. Next, the feature extraction is done for the extracted keywords and semantic words using holoentropy to attain information statistics, which results in the attainment of maximum related information. Here, two categories of holoentropy features are extracted: joint holoentropy and cross holoentropy. These extracted features of entire keywords are finally subjected to a hybrid classifier, which merges the beneficial concepts of neural network (NN), and deep belief network (DBN). For improving the performance of sentiment classification, modification is done by inducing the idea of a modified rider optimization algorithm (ROA), so-called new steering updated ROA (NSU-ROA) into NN and DBN for weight update. Hence, the average of both improved classifiers will provide the classified sentiment as positive, negative or neutral from the reviews of newspaper articles effectively.FindingsThree data sets were considered for experimentation. The results have shown that the developed NSU-ROA + DBN + NN attained high accuracy, which was 2.6% superior to particle swarm optimization, 3% superior to FireFly, 3.8% superior to grey wolf optimization, 5.5% superior to whale optimization algorithm and 3.2% superior to ROA-based DBN + NN from data set 1. The classification analysis has shown that the accuracy of the proposed NSU − DBN + NN was 3.4% enhanced than DBN + NN, 25% enhanced than DBN and 28.5% enhanced than NN and 32.3% enhanced than support vector machine from data set 2. Thus, the effective performance of the proposed NSU − ROA + DBN + NN on sentiment analysis of newspaper articles has been proved.Originality/valueThis paper adopts the latest optimization algorithm called the NSU-ROA to effectively recognize the sentiments of the newspapers with NN and DBN. This is the first work that uses NSU-ROA-based optimization for accurate identification of sentiments from newspaper articles.

The temporal and spatial information of electroencephalogram (EEG) are essential for the emotion recognition model to learn the discriminative features. Hence, we propose a novel hybrid spatial-temporal feature fusion neural network (STFFNN) to extract the discriminative features and integrate complementary information. The generated power topographic maps, which capture dependencies among the electrodes, are fed to convolutional neural network (CNN) for spatial feature learning. Furthermore, instance normalizations (INs) and batch normalizations (BNs) within the CNN are appropriately combined to alleviate the individual difference and preserve the domain-invariant information. Meanwhile, a feedforward network is adopted for temporal feature learning. Due to the high dimensionality of EEG features, we propose a grid-search-based configurational optimization method to robustly reduce the dimensionality. Finally, inspired by the multimodal fusion strategies that leverage the complementarity of data to obtain more robust predictions, we utilize a bidirectional long short-term memory (Bi-LSTM) network for temporal and spatial feature fusion. To validate the effectiveness of the proposed method, the tenfold cross-validation experiments and subject-dependent experiments are both conducted on the DEAP database. The experimental results demonstrate that the proposed method achieves outstanding performance in emotion recognition with arousal and valence level.

Emotion recognition from EEG signals plays a significant role in affective computing, and it enables the systems to be aware of the human emotional state. This research proposes a novel methodology for zero-shot learning in multi-class emotion recognition from EEG signals, which address the issues like data scarcity, class imbalance, and generalization. A technique starting with a dataset acquisition method makes use of EEG-GANs in order to produce synthetic samples to balance class imbalance for poorly represented emotional classes. The introduction of DWC-TFS is innovatively used: the dynamic patterns in connectivity have been captured appropriately and the temporally noisy has reduced in the study. In order to extract features, DFCNs are applied in modeling time-varying relationships between EEG channels, keeping important temporal dependencies intact. For feature selection, Neuro-Symbolic Feature Selection is used by incorporating symbolic reasoning with deep learning in order to choose the most discriminative features. The classification task is done using a T-ZSC Transformer-Based Zero-Shot Classifier. The model will generalize across unseen emotional classes through the use of semantic embeddings and self-attention mechanisms. The proposed methodology achieves an accuracy of 96.2%, demonstrating robust performance in emotion recognition. This integrated approach offers a powerful solution for addressing challenges in EEG-based emotion recognition, ensuring improved class balance, generalization, and accuracy in real-world applications.

This paper adapts a novel model, deep belief nets (DBN), to extract entity mentions in Chinese documents. Our experiments were designed to develop entity detection system and entity categorization system applying DBN, to complete entity extraction. Our results exhibit how the depth of architecture and quantity of unit in hidden layer of DBN influence the performance. In DBN Systems, token labels are produced independently and DBN does not concerned what the labels of surrounding token are. Viterbi algorithm is a good solution to overcome this issue. It can find the most likely probability label path to make DBN to be more suitable for entity detection. Furthermore, this paper demonstrates DBN is proper model for our tasks and its results are better than Support Vector Machine (SVM), Artificial Neural Network (ANN) and Conditional Random Field (CRF).

Traditional speech emotion recognition (SER) evaluations have been performed merely on a speaker-independent condition; some of them even did not evaluate their result on this condition. This paper highlights the importance of splitting training and test data for SER by script, known as sentence-open or text-independent criteria. The results show that employing sentence-open criteria degraded the performance of SER. This finding implies the difficulties of recognizing emotion from speech in different linguistic information embedded in acoustic information. Surprisingly, text-independent criteria consistently performed worse than speaker+text-independent criteria. The full order of difficulties for splitting criteria on SER performances from the most difficult to the easiest is text-independent, speaker+text-independent, speaker-independent, and speaker+text-dependent. The gap between speaker+text-independent and text-independent was smaller than other criteria, strengthening the difficulties of recognizing emotion from speech in different sentences.