Discriminative auditory features for robust speech recognition

This paper proposes a neural mechanism motivated system to extract noise resistant features for robust speech recognition. We use nonnegative matrix factorization to construct two layers of auditory neurons which captures the essence of speech patterns. The responses of these neurons to speech are further processed to form an auditory neural cepstral coefficient (ANCC) representation for speech recognition. We test the robustness of ANCC feature on a 51-word corpus, with recognizers trained on clean speech in noisy conditions. Compared with MFCC, ANCC shows less performance degradation and achieves satisfactory recognition accuracies in both non-stationary noise and high noise level conditions.

This paper presents an investigation of robust features for speech recognition in three different noisy environments. The state-of art Mel-frequency cepstral coefficients were extensively explored in additive, convolutive and reverberant environments. These environments have captured the interest of many researches in speech recognition systems. We evaluate robust speech recognition results on the TI-DIGIT database. Significant word error rate reductions were observed in the connected digit recognition experiments. The recognition experiments vindicate the robustness of Mel-frequency cepstral coefficient with dynamic features and cepstral mean normalization in hostile environments, especially additive and reverberant noise

Combined speech enhancement and auditory modelling for robust distributed speech recognition

Most current speech recognition systems are built upon a single type of model, e.g. an HMM or certain type of segment based model, and furthermore typically employs only one type of acoustic feature e.g. MFCCs and their variants. This entails that the system may not be robust should the modeling assumptions be violated. Recent research efforts have investigated the use of multi-scale/multi-band acoustic features for robust speech recognition. This paper described a multi-model approach as an alternative and complement to the multi-feature approaches. The multi-model approach seeks a combination of different types of acoustic models, thereby integrating the capabilities of each individual model for capturing discriminative information. An example system built upon the combination of the standard HMM technique with a segment-based modeling technique was implemented. Experiments for both isolated-word and continuous speech recognition have shown improved performances over each of the individual models considered in isolation.

Exploring joint equalization of spatial-temporal contextual statistics of speech features for robust speech recognition

Intra-speaker variations due to perceptually induced stress or emotion adversely affect speech recognition system performance. In this paper, we combine auditory based (Mel frequency cepstral coefficients and linear predictive cepstral coefficients) features and articulatory based (voicedness) features for robust speech recognition. Voicedness features are derived using linear and teager energy operator (TEO) based nonlinear fast Fourier transform (FFT) spectra. Nonlinear properties are analyzed in both the time and frequency domains. In addition, we investigate the sensitivity of all these FFT spectra to stress and observe the performance of individual FFT spectra. The system is tested using stressed speech data from the speech under simulated and actual stress (SUSAS) database. The results show that articulatory based features help to improve the system performance. Furthermore, significant performance improvement has been observed when using the FFT spectrum which is less sensitive to stress

In this paper, we propose two estimators for the autocorrelation sequence of a periodic signal in additive noise. Both estimators are formulated employing tables which contain all the possible products of sample pairs in a speech signal frame. The first estimator is based on a pitch-synchronous averaging. This estimator is statistically analyzed and we show that the signal-to-noise ratio (SNR) can be increased up to a factor equal to the number of available periods. The second estimator is similar to the former one but it avoids the use of those sample products more likely affected by noise. We prove that, under certain conditions, this estimator can remove the effect of an additive noise in a statistical sense. Both estimators are employed to extract mel frequency cepstral coefficients (MFCCs) as features for robust speech recognition. Although these estimators are initially conceived for voiced speech frames, we extend their application to unvoiced sounds in order to obtain a coherent feature extractor. The experimental results show the superiority of the proposed approach over other MFCC-based front-ends such as the higher-lag autocorrelation spectrum estimation (HASE), which also employs the idea of avoiding those autocorrelation coefficients more likely affected by noise.

S.5492-5495

Recently, a new auditory-based feature extraction algorithm for robust speech recognition in noisy environments was proposed. The new features are derived by mimicking closely the human peripheral auditory process and the filters in the outer ear, middle ear, and inner ear are obtained from psychoacoustics literature with some manual adjustments. In this paper, we extend the auditory-based feature extraction algorithm and propose to further train the auditory-based filters through training. Using the data-driven approach, we optimize the filters by minimizing the subsequent recognition errors on a task. One significant contribution over similar efforts in the past (generally under the name of discriminative feature extraction) is that we make no assumption on the parametric form of the auditory-based filters. Instead, we only require the filters to be triangular-like: the filter weights have a maximum value in the middle and then monotonically decrease to both ends. Discriminative training of these constrained auditory-based filters leads to improved performance. Furthermore, we study the combined training procedure for both feature and acoustic model parameters. Our experiments show that the best performance can be obtained in a sequential procedure under the unified framework of MCE/GPD.

Discriminative multi-layer illumination-robust feature extraction for face recognition

One key challenge in facial expression recognition (FER) is the extraction of discriminative features from critical facial regions. Because of their promising ability to learn discriminative features, visual attention mechanisms are increasingly used to address pattern recognition problems. This paper presents a novel multiple attention network that simulates humans' coarse-to-fine visual attention to improve expression recognition performance. In the proposed network, a region-aware sub-net (RASnet) learns binary masks for locating expression-related critical regions with coarse-to-fine granularity levels and an expression recognition sub-net (ERSnet) with a multiple attention (MA) block learns comprehensive discriminative features. Embedded in the convolutional layers, the MA block fuses diversified attention using the learned masks from the RASnet. The MA block contains a hybrid attention branch with a series of sub-branches, where each sub-branch provides region-specific attention. To explore the complementary benefits of diversified attention, the MA block also has a weight learning branch that adaptively learns the contributions of the different critical regions. Experiments have been carried out on two publicly available databases, RAF and CK+, and the reported accuracies are 85.69% and 96.28%, respectively. The results indicate that our method achieves competitive or better performance than state-of-the-art methods.

Discovering a discriminative feature representative together with a suitable distance measure is the key for a successful speaker recognition system. In this paper, we propose a new approach for automatic speaker verification. The main contribution of the paper is the extraction of discriminative speaker features using non-negative matrix factorization (NMF) decomposition in the GMM mean space, and the use of cosine-distance measure for speaker classification. With the decomposition, the speaker space is represented by the pattern components while a speaker can be characterized by a coefficient vector representing a specific localization in the space. We validate the proposed approach on the 10-second training and 10-second testing condition constructed from 863 Putonghua (Mandarin) corpus. Relative 10.57% and 26.11% improvements compared to the conventional GMM-UBM system have been achieved for female and male trials respectively.

The problem of ultra-high dimensionality small samples exists in the face data. In order to obtain discriminative features with higher generalization ability and improve the face recognition rate, a weighted maximum margin criterion (WMMC) method is presented for extracting discriminative features of face data. The method considers that the actual inter-class scatter is less than the inter-class scatter reflected by the inter-class scatter matrix calculated with limited training samples, and the actual intra-class scatter is more than the intra-class scatter reflected by the intra-class scatter matrix calculated with limited training samples, the WMMC function is obtained by recalculating the inter-class scatter matrix and intra-class scatter matrix. Experimental results using minimum distance (MD) and k-nearest neighbor (kNN) classifiers on ORL and Yale face datasets show that WMMC extracts discriminative features with higher generalization ability and significantly improves face recognition rate compared to maximum margin criterion (MMC).KeywordsFeature extractionWeighted maximum margin criterion (WMMC)Linear discriminant analysis (LDA)Face recognition

Recently, attention based sequence-to-sequence model become a research hotspot in speech recognition. The attention model has the problem of slow convergence and poor robustness. In this paper, a model that jointed a bottleneck feature extraction network and attention model is proposed. The model is composed of a Deep Belief Network as bottleneck feature extraction network and an attention-based encoder-decoder model. DBN can store the priori information from Hidden Markov Model so that increasing convergence speed of and enhancing both robustness and discrimination of features. Attention model utilizes the temporal information of feature sequence to calculate the posterior probability of phoneme. Then the number of stack recurrent neural network layers in attention model is reduced in order to decrease the calculation of gradient. Experiments in the TIMIT corpus showed that the phoneme error rate is 17.80% in test set, the average training iteration decreased 52%, and the number of training iterations decreased from 139 to 89. The word error rate of WSJ eval92 is 12.9% without any external language model.

Human action analytics has attracted a lot of attention for decades in computer vision. It is important to extract discriminative spatio-temporal features to model the spatial and temporal evolutions of different actions. In this paper, we propose a spatial and temporal attention model to explore the spatial and temporal discriminative features for human action recognition and detection from skeleton data. We build our networks based on the recurrent neural networks with long short-term memory units. The learned model is capable of selectively focusing on discriminative joints of skeletons within each input frame and paying different levels of attention to the outputs of different frames. To ensure effective training of the network for action recognition, we propose a regularized cross-entropy loss to drive the learning process and develop a joint training strategy accordingly. Moreover, based on temporal attention, we develop a method to generate the action temporal proposals for action detection. We evaluate the proposed method on the SBU Kinect Interaction data set, the NTU RGB + D data set, and the PKU-MMD data set, respectively. Experiment results demonstrate the effectiveness of our proposed model on both action recognition and action detection.