Amplitude modulation spectrogram based features for robust speech recognition in noisy and reverberant environments

This paper presents an investigation of robust features for speech recognition in three different noisy environments. The state-of art Mel-frequency cepstral coefficients were extensively explored in additive, convolutive and reverberant environments. These environments have captured the interest of many researches in speech recognition systems. We evaluate robust speech recognition results on the TI-DIGIT database. Significant word error rate reductions were observed in the connected digit recognition experiments. The recognition experiments vindicate the robustness of Mel-frequency cepstral coefficient with dynamic features and cepstral mean normalization in hostile environments, especially additive and reverberant noise

Combined speech enhancement and auditory modelling for robust distributed speech recognition

Most current speech recognition systems are built upon a single type of model, e.g. an HMM or certain type of segment based model, and furthermore typically employs only one type of acoustic feature e.g. MFCCs and their variants. This entails that the system may not be robust should the modeling assumptions be violated. Recent research efforts have investigated the use of multi-scale/multi-band acoustic features for robust speech recognition. This paper described a multi-model approach as an alternative and complement to the multi-feature approaches. The multi-model approach seeks a combination of different types of acoustic models, thereby integrating the capabilities of each individual model for capturing discriminative information. An example system built upon the combination of the standard HMM technique with a segment-based modeling technique was implemented. Experiments for both isolated-word and continuous speech recognition have shown improved performances over each of the individual models considered in isolation.

Intra-speaker variations due to perceptually induced stress or emotion adversely affect speech recognition system performance. In this paper, we combine auditory based (Mel frequency cepstral coefficients and linear predictive cepstral coefficients) features and articulatory based (voicedness) features for robust speech recognition. Voicedness features are derived using linear and teager energy operator (TEO) based nonlinear fast Fourier transform (FFT) spectra. Nonlinear properties are analyzed in both the time and frequency domains. In addition, we investigate the sensitivity of all these FFT spectra to stress and observe the performance of individual FFT spectra. The system is tested using stressed speech data from the speech under simulated and actual stress (SUSAS) database. The results show that articulatory based features help to improve the system performance. Furthermore, significant performance improvement has been observed when using the FFT spectrum which is less sensitive to stress

Recently, Li et al. proposed a new auditory feature for robust speech recognition in noise environments. The new feature was derived by mimicking closely the function of human auditory process. Several filters were used to model the outer ear, middle ear, and cochlea, and the initial filter parameters and shapes were obtained from crude psychoacoustics results, experience, or experiments. Although one may adjust the feature parameters by hand to get better performance, the resulting feature parameters still may not be optimal in the sense of minimal recognition errors, especially for different tasks. To further improve the auditory feature, in this paper we apply discriminative training to optimize the auditory feature parameters with some guidance from psychoacoustic evidence but otherwise in a data-driven approach so as to minimize the recognition errors. One significant contribution over similar efforts in the past, such as discriminative feature extraction, is that we make no assumption on the parametric form of the auditory filters. Instead, we only require the filters to be smooth and triangular-like as suggested by psychoacoustics research. Our approach is evaluated on the Aurora database and achieves a word error reduction of 19.2%.

This paper proposes a neural mechanism motivated system to extract noise resistant features for robust speech recognition. We use nonnegative matrix factorization to construct two layers of auditory neurons which captures the essence of speech patterns. The responses of these neurons to speech are further processed to form an auditory neural cepstral coefficient (ANCC) representation for speech recognition. We test the robustness of ANCC feature on a 51-word corpus, with recognizers trained on clean speech in noisy conditions. Compared with MFCC, ANCC shows less performance degradation and achieves satisfactory recognition accuracies in both non-stationary noise and high noise level conditions.

Exploring joint equalization of spatial-temporal contextual statistics of speech features for robust speech recognition

In this paper, we propose two estimators for the autocorrelation sequence of a periodic signal in additive noise. Both estimators are formulated employing tables which contain all the possible products of sample pairs in a speech signal frame. The first estimator is based on a pitch-synchronous averaging. This estimator is statistically analyzed and we show that the signal-to-noise ratio (SNR) can be increased up to a factor equal to the number of available periods. The second estimator is similar to the former one but it avoids the use of those sample products more likely affected by noise. We prove that, under certain conditions, this estimator can remove the effect of an additive noise in a statistical sense. Both estimators are employed to extract mel frequency cepstral coefficients (MFCCs) as features for robust speech recognition. Although these estimators are initially conceived for voiced speech frames, we extend their application to unvoiced sounds in order to obtain a coherent feature extractor. The experimental results show the superiority of the proposed approach over other MFCC-based front-ends such as the higher-lag autocorrelation spectrum estimation (HASE), which also employs the idea of avoiding those autocorrelation coefficients more likely affected by noise.

In this paper, alternative dynamic features for speech recognition are proposed. The goal of this work is to improve speech recognition accuracy by deriving the representation of distinctive dynamic characteristics from a speech spectrum. This work was inspired by two temporal dynamics of a speech signal. One is the highly non-stationary nature of speech, and the other is the inter-frame change of a speech spectrum. We adopt the use of a sub-frame spectrum analyzer to capture very rapid spectral changes within a speech analysis frame. In addition, we attempt to measure spectral fluctuations of a more complex manner as opposed to traditional dynamic features such as delta or double-delta. To evaluate the proposed features, speech recognition tests over smartphone environments were conducted. The experimental results show that the feature streams simply combined with the proposed features are effective for an improvement in the recognition accuracy of a hidden Markov model–based speech recognizer.

In this paper, we compare several approaches for the extraction of modulation frequency features from speech signal using a phoneme recognition system. The general framework in these approaches is to decompose the speech signal into a set of sub-bands. Amplitude modulations (AM) in the sub-band signal are used to derive features for automatic speech recognition (ASR). Then, we propose a feature extraction technique which uses autoregressive models (AR) of sub-band Hilbert envelopes in relatively long segments of speech signal. AR models of Hilbert envelopes are derived using frequency domain linear prediction (FDLP). Features are formed by converting the FDLP envelopes into static and dynamic modulation frequency components. In the phoneme recognition experiments using the TIMIT database, the FDLP based modulation frequency features provide significant improvements compared to other techniques (average relative improvement of 7.5 % over the base-line features). Furthermore, a detailed analysis is performed to determine the relative contribution of various processing stages in the proposed technique.

CENSREC-4 evaluation framework has been distributed for evaluating distant-talking speech under various noisy and reverberant environments. It however has not been evaluated how variable noisy and reverberant features in this contains. We thus try to evaluate CENSREC-4 with our designed noisy and reverberant criteria based on PESQ and acoustic parameters. We specifically focus on criteria to represent the difficulty of noisy and reverberant speech recognition, and also confirm why it is difficult to easily evaluate the recognition accuracy in a part of CENSREC-4 corpus with our proposed noisy and reverberant criteria. We first designed the noisy and reverberant criteria using the relationship among the D value, the PESQ, and the ASR performance. We then tried to estimate the recognition accuracy in various noisy and reverberant environments with CENSREC-4. We carried out evaluation experiments to confirm the difficulty to easily evaluate the recognition accuracy in a part of CENSREC-4 courpus0. As a result of evaluation experiments, we confirmed that it was difficult to estimate the accuracy of noisy and reverberant speech recognition in heavy noisy and reverberant environment with CENSREC-4. We therefore confirmed that CENSREC-4 contained very challenging and variable noisy and reverberant data.

A mathematical framework based on maximum likelihood stochastic matching is proposed to perform feature and model compensation for robust speech recognition. Speech recognition is often formulated as a matching problem between the feature vectors extracted from a test utterance and a set of speech models or patterns obtained from some training corpra. It is well known that a speech recognizer often degrades in performance when the testing data are not acoustically similar to the training data. One way to improve is to find features that are invariant under all acoustic conditions and distortions. Some form of compensation is often required. The proposed stochastic matching approach assumes a structure or a form of the feature and/or model transformations. Together with a set of nuisance parameters, the transformations approximate the distortion in the test utterance. To decrease the acoustic mismatch between a test utterance and a given set of speech models, e.g., hidden Markov models, the stochastic matching algorithm estimates the nuisance parameters and then applies the feature/model transformations during speech recognition. Simple channel distortion can be approximated with linear transformations. For more complicated distortions, such as environmental, speaker, and combined mismatches, nonlinear compensating transformations are needed. These compensations give a significant performance improvement in speech recognition over the systems without them when utterances are affected by additive ambient noises and convolutional channel distortions.

Typical hands-free voice capturing relies on recording a distant speaker using an array of microphones. When speech signal is recorded in a reverberant environment, the microphones capture speech corrupted by room reverberation. Increasing the distance between the speaker and recording microphones, the power ratio of the direct and late room reverberation signals is decreased, which in turn decreases the accuracy of automatic speech recognition. This paper presents a multichannel Wiener filter which exploits geometric information about early room reflection paths in order to reduce late reverberation at the filter output. As a result of such pre-processing, the accuracy of automatic speech recognition systems is improved. The presented word error rate (WER) results for simulated rooms with different reverberation times indicate that the proposed rake multichannel Wiener filtering provides an improvement in accuracy of automatic speech recognition from distance in reverberant environments. The gain in terms of word error rate achieved in considered acoustic conditions reaches several percent.

This paper describes an attempt to extract peripheral features of a point c(t/sub i/,q/sub j/) on a time-quefrency (TQ) pattern by observing n/spl times/n neighborhoods of the point, and then to incorporate these peripheral features into the MFCC-based feature extractor of a speech recognition system as a replacement to dynamic features. In the design of the feature extractor, firstly, the orthogonal bases extracted directly from speech data by using the Karhunen-Loeve transform (KLT) of 7/spl times/3 blocks on a TQ pattern are adopted as the peripheral features, then, the upper two primal bases are selected and simplified in the form of /spl utri//sub t/-operator and /spl utri//sub q/-operator. The proposed feature-set of MFCC and peripheral features shows significant improvements in comparison with the standard feature-set of MFCC and dynamic features in experiments with an HMM-based automatic speech recognition (ASR) system. The reason for the increased performance is discussed in terms of minimal-pair tests.

In previous work, we proposed a new speech recognition technique that generates a smooth speech trajectory from hidden Markov models (HMMs) by maximizing likelihood subject to the constraints that exist between static and dynamic speech features. This paper presents a theoretical analysis of this method. We show that the approach used to generate the smoothed trajectory is equivalent to a Kalman filter. This result demonstrates that there is a strong relationship between the dynamics of delta features (and delta-delta features) in HMM-based speech recognition and Kalman filter dynamics.