Discriminative piecewise linear transformation based on deep learning for noise robust automatic speech recognition

The probabilistic optimum filtering (POF) algorithm is a piece wise linear transformation of the noisy speech feature space into the clean speech feature space. In this work we extend the POF algorithm to allow a more accurate way to select noisyto-clean feature mappings, by allowing different combinations of speech and noise to have combination-specific mappings selected depending on the observation. This is especially important in nonstationary environments, where different noise segments will result in different observations in the noisy feature space. Experimental results using stationary and nonstationary noises show the effectiveness of the proposed technique compared to the old approach. We also explored the use of the extended POF method to train a map with multiple noises in order to gain generalization over different noise types and be able to tackle unknown noise environments.

In this paper we investigate stereo-based stochastic mapping (SSM) with context for the noise robustness of automatic speech recognition, especially under unseen conditions. Probabilistic PCA (PPCA) is used in the SSM framework to reduce the high dimensionality of the noisy speech features with context and derive an eigen representation in the noisy feature space for the prediction of clean features. To reduce the computational cost in training, an approximation by single-pass re-training is considered for the estimation of joint GMM. We also show that the SSM estimate under the minimum mean square error (MMSE) in a space where low dimensional representation of clean speech and uncorrelated additive noise can be assumed is related to the subspace speech enhancement. Experiments on large vocabulary continuous speech recognition tasks observe gains from the proposed approach under the conditions with seen, unseen and real noise.

In this paper, a set of feature vector normalization methods based on the minimum mean square error (MMSE) criterion and stereo data is presented. They include multi-environment model-based linear normalization (MEMLIN), polynomial MEMLIN (P-MEMLIN), multi-environment model-based histogram normalization (MEMHIN), and phoneme-dependent MEMLIN (PD-MEMLIN). Those methods model clean and noisy feature vector spaces using Gaussian mixture models (GMMs). The objective of the methods is to learn a transformation between clean and noisy feature vectors associated with each pair of clean and noisy model Gaussians. The direct approach to learn the transformation is by using stereo data; that is, noisy feature vectors and the corresponding clean feature vectors. In this paper, however, a nonstereo data based training procedure, is presented. The transformations can be modeled just like a bias vector (MEMLIN), or by using a first-order polynomial (P-MEMLIN) or a nonlinear function based on histogram equalization (MEMHIN). Further improvements are obtained by using phoneme-dependent bias vector transformation (PD-MEMLIN). In PD-MEMLIN, the clean and noisy feature vector spaces are split into several phonemes, and each of them is modeled as a GMM. Those methods achieve significant word error rate improvements over others that are based on similar targets. The experimental results using the SpeechDat Car database show an average improvement in word error rate greater than 68% in all cases compared to the baseline when using the original clean acoustic models, and up to 83% when training acoustic models on the new normalized feature space

An effective way to increase noise robustness in automatic speech recognition (ASR) systems is feature enhancement based on an analytical distortion model that describes the effects of noise on the speech features. One of such distortion models that has been reported to achieve a good trade-off between accuracy and simplicity is the masking model. Under this model, speech distortion caused by environmental noise is seen as a spectral mask and, as a result, noisy speech features can be either reliable (speech is not masked by noise) or unreliable (speech is masked). In this paper, we present a detailed overview of this model and its applications to noise robust ASR. Firstly, using the masking model, we derive a spectral reconstruction technique aimed at enhancing the noisy speech features. Two problems must be solved in order to perform spectral reconstruction using the masking model: (1) mask estimation, i.e. determining the reliability of the noisy features, and (2) feature imputation, i.e. estimating speech for the unreliable features. Unlike missing data imputation techniques where the two problems are considered as independent, our technique jointly addresses them by exploiting a priori knowledge of the speech and noise sources in the form of a statistical model. Secondly, we propose an algorithm for estimating the noise model required by the feature enhancement technique. The proposed algorithm fits a Gaussian mixture model to the noise by iteratively maximising the likelihood of the noisy speech signal so that noise can be estimated even during speech-dominating frames. A comprehensive set of experiments carried out on the Aurora-2 and Aurora-4 databases shows that the proposed method achieves significant improvements over the baseline system and other similar missing data imputation techniques.

Sensitive information such as credit card information, username, password and social security number etc, can be stolen using a fake page that imitates trusted website is called phishing. The attacker designs a similar webpage either by copying or making small manipulation to the legitimate page so that the online user cannot distinguish the legitimate and fake websites. A Deep Neural Network (DNN) was introduced to detect the phishing Uniform Resource Locator (URL). Initially, a 30-dimension feature vector was constructed based on URL-based features, Hypertext Markup Language (HTML)-based features and domain-based features. These features were processed in DNN to detect the phishing URL. However, the irrelevant, redundant and noisy features in the dataset increase the complexity of DNN classifier. So the feature selection is required for efficient phishing attack detection. But feature selection is a time-consuming process since it is an independent process. So in this paper, a feature vector is generated by DNN itself using Stacked Denoise Auto Encoder (SDAE). Moreover, the noisy data such as missing features affect the efficiency of phishing detection so the SDAE is trained to reconstruct a clean input feature vector. The initial input feature vector is corrupted by setting some feature vectors as zero. Then the corrupted feature vector is then mapped with basic auto encoder, to a hidden representation from which the input feature vector is reconstructed. The reconstructed features are given as input to DNN which selects the most relevant features and predicts the phishing URL. Hence the sparse feature representation of SDAE increases the classification accuracy of DNN. The experiments are conducted in Ham, Phishing Corpus and Phishload datasets in terms of accuracy, precision, recall and F-measure to prove the effectiveness of DNN-SDAE.

Since the introduction of deep neural network (DNN)-based acoustic model, robust automatic speech recognition using DNN are being in research. Especially in model adaptation, the techniques utilizing auxiliary context features is known to be a promising technique. Recently, we proposed a technique which is called two-stage noise-aware training (TSNAT). The key idea of TS-NAT is to let the DNN clarify the relationship among noise estimate, noisy features and phonetic target through clean feature representation. However, although TS-NAT enhances the robustness of the DNN, we cannot be certain whether TS-NAT describes the clean feature representation sufficiently. In this paper, we extend TS-NAT using true noise feature and various DNN training techniques. It has been shown that the proposed technique outperforms the conventional DNN-based techniques on Aurora5-task and mismatched noise conditions.

The histogram equalization approach is an efficient feature normalization technique for noise robust automatic speech recognition. However, it suffers from performance degradation when some fundamental conditions are not satisfied in the test environment. To remedy these limitations of the original histogram equalization methods, class-based histogram equalization approach has been proposed. Although this approach showed substantial performance improvement under noise environments, it still suffers from performance degradation due to the overfitting problem when test data are insufficient. To address this issue, the proposed histogram equalization technique employs the Bayesian estimation method in the test cumulative distribution function estimation. It was reported in a previous study conducted on the Aurora-4 task that the proposed approach provided substantial performance gains in speech recognition systems based on the acoustic modeling of the Gaussian mixture model-hidden Markov model. In this work, the proposed approach was examined in speech recognition systems with deep neural network-hidden Markov model (DNN-HMM), the current mainstream speech recognition approach where it also showed meaningful performance improvement over the conventional maximum likelihood estimation-based method. The fusion of the proposed features with the mel-frequency cepstral coefficients provided additional performance gains in DNN-HMM systems, which otherwise suffer from performance degradation in the clean test condition.

We propose a novel exemplar-based feature enhancement method for automatic speech recognition which uses coupled dictionaries: an input dictionary containing atoms sampled in the modulation (envelope) spectrogram domain and an output dictionary with atoms in the Mel or full-resolution frequency domain. The input modulation representation is chosen for its separation properties of speech and noise and for its relation with human auditory processing. The output representation is one which can be processed by the ASR back-end. The proposed method was investigated on the AURORA-2 and AURORA-4 databases and improved word error rates (WER) were obtained when compared to the system which uses Mel features in the input exemplars. The paper also proposes a hybrid system which combines the baseline and the proposed algorithm on the AURORA-2 database which in turn also yielded improvement over both the algorithms.

Biometrics offers more security and convenience than traditional methods of identification. Recently, DNN has become a means of a more reliable and efficient authentication scheme. In this work, we compare two modern teaching methods: these two methods are methods based on the Gaussian mixture model (GMM) (denoted by the GMM i-vector) and methods based on deep neural networks (DNN) (denoted as the i-vector DNN). The results show that the DNN system with an i-vector is superior to the GMM system with an i-vector for various durations (from full length to 5s). DNNs have proven to be the most effective features for text-independent speaker verification in recent studies. In this paper, a new scheme is proposed that allows using DNN when checking text using hints in a simple and effective way. Experiments show that the proposed scheme reduces EER by 24.32% compared with the modern method and is evaluated for its reliability using noisy data, as well as data collected in real conditions. In addition, it is shown that the use of DNN instead of GMM for universal background modeling leads to a decrease in EER by 15.7%.

Voice Conversion (VC) requires an alignment of the spectral features before learning the mapping function, due to the speaking rate variations across the source and target speakers. To address this issue, the idea of training two parallel networks with the use of speaker-independent representation was proposed. In this paper, we explore the unsupervised Gaussian Mixture Model (GMM) posteriorgram as a speaker-independent representation. However, in the GMM posteriorgram, the same phonetic information gets spread across more than one component due to the speaking style variations across the speakers. In particular, this spread is limited to a group of neighboring components for a given phone. We propose to share the posterior probability of each component with the limited number of neighboring components that are sorted based on the Kullback-Leibler (KL) divergence. We propose to employ a Deep Neural Network (DNN) and a Generative Adversarial Network (GAN)-based framework to measure the effectiveness of the proposed Inter Mixture Weighted GMM (IMW GMM) posteriorgram on the Voice Conversion Challenge (VCC) 2016 database. The relative improvement of 13.73%, and 5.25% is obtained with the proposed IMW GMM posteriorgram w.r.t. the GMM posteriorgram for the speech quality and the speaker similarity of the converted voices, respectively.

In rarefied gas flows, discontinuity phenomena such as velocity slip and temperature jump commonly appear in the gas layer adjacent to a solid boundary. Due to the physical complexity of the interactions at the gas-solid interface, particularly in the case of systems with local nonequilibrium state, boundary models with limited number of parameters cannot completely describe the reflection of gas molecules at the boundary. In this work, the Gaussian mixture (GM) model, which is an unsupervised machine learning technique, is employed to construct a statistical gas-solid surface scattering model based on the collisional data obtained from molecular dynamics (MD) simulations. The GM model is applied to study Couette flow for different inert gases (Ar and He) confined between two parallel infinite gold walls at different temperatures. A direct comparison between the results obtained from the GM model and the Cercignani-Lampis-Lord (CLL) scattering kernel against the MD collisional data in terms of the distribution of the predicted postcollisional velocities, and accommodation coefficients has shown that the results from the GM model are an excellent match with the MD results outperforming the CLL scattering kernel. As an example, for He gas, while the predicted energy accommodation coefficient by the CLL model is more than two times higher than the MD predictions, the value computed by the GM model is in excellent agreement with the MD results. This superior performance of the GM model confirms its high potential to derive a generalized boundary condition in systems encountered with highly nonequilibrium and complex gas flow conditions.

This paper proposes stereo-based speech feature enhancement using dictionary learning. Instead of posterior values obtained by a Gaussian mixture as in other methods, we use sparse weight vectors and their variants as an alternative noisy speech feature representation. This paper also provides an efficient algorithm that can be applied to large-scale speech processing. We show the effectiveness of the proposed approach by using a middle vocabulary noisy speech recognition task based on WSJ, which was provided by the 2nd CHiME Speech Separation and Recognition Challenge.

While deep neural networks (DNNs) have become the dominant acoustic model (AM) for speech recognition systems, they are still dependent on Gaussian mixture models (GMMs) for alignments both for supervised training and for context dependent (CD) tree building. Here we explore bootstrapping DNN AM training without GMM AMs and show that CD trees can be built with DNN alignments which are better matched to the DNN model and its features. We show that these trees and alignments result in better models than from the GMM alignments and trees. By removing the GMM acoustic model altogether we simplify the system required to train a DNN from scratch.

Sparse representations have been found to provide high classification accuracy in many fields. Their drawback is the high computational load. In this work, we propose a novel cascaded classifier structure to speed up the decision process while utilizing sparse signal representation. In particular, we apply the cascaded decision process for noise robust automatic speech recognition task. The cascaded decision process is implemented using a feedforward neural network (NN) and time sparse versions of a non-negative matrix factorization (NMF) based sparse classification method of [1]. The recognition accuracy of our cascade is among the three best in the recent CHiME2013 benchmark and obtains six times faster the accuracy of NMF alone as in [1].

As with many approaches to noise robust automatic speech recognition (ASR) the benefits of spectral subtraction tend to diminish as noise levels in the order of 0 dB are approached. Whilst the majority of related work focuses on reducing magnitude errors a number of new approaches addressing the often overlooked, additional sources of error have appeared in the literature in recent years. Relatively lacking in the literature, however, is an empirical assessment which compares the effects of each error when noisy speech is processed by spectral subtraction. Such studies are vital in order to appreciate the potential penalty in performance when sources of error are overlooked. The objective in this paper is to assess, through ASR, the performance penalty associated with each source of error when noisy speech is treated with spectral subtraction. Experimental evidence based on two standard European databases and ASR protocols illustrates that, perhaps contrary to popular belief, for noise levels in the order of 0 dB and below, these often overlooked sources of error can lead to non-negligible degradations in performance. Whilst not a new idea, here the original emphasis is a thorough assessment that empirically highlights both the fundamental limitations and potential benefit of including the full complement of errors in the spectral subtraction model.