Speaker normalization in noisy environments using subglottal resonances

This paper proposes a novel feature-space VTLN (vocal tract length normalization) method that models frequency warping as a linear interpolation of contiguous Mel filter-bank energies. The presented technique aims to reduce the distortion in the Mel filter-bank energy estimation due to the harmonic composition of voiced speech intervals and DFT (discrete Fourier transform) sampling when the central frequency of band-pass filters is shifted. This paper also proposes an analytical maximum likelihood (ML) method to estimate the optimal warping factor in the cepstral space. The presented interpolated filter-bank energy-based VTLN leads to relative reductions in WER (word error rate) as high as 11.2% and 7.6% when compared with the baseline system and standard VTLN, respectively, in a medium-vocabulary continuous speech recognition task. Also, the proposed VTLN scheme can provide significant reductions in WER when compared with state-of-the-art VTLN methods based on linear transforms in the cepstral feature-space. The warping factor estimated with the proposed VTLN approach shows more dependence on the speaker and more independence of the acoustic-phonetic content than the warping factor resulting from standard and state-of-the-art VTLN methods. Finally, the analytical ML-based optimization scheme presented here achieves almost the same reductions in WER as the ML grid search version of the technique with a computational load 20 times lower.

This paper shows the impact of speaker normalization techniques, such as vocal tract length normalization (VTLN) and speaker-adaptive training (SAT), prior to discriminant feature space transforms, such as LDA (linear discriminant analysis). We demonstrate that removing the inter-speaker variability by using speaker compensation methods results in improved discrimination as measured by the LDA eigenvalues and also in improved classification accuracy (as measured by the word error rate). Experimental results on the SPINE (speech in noisy environments) database indicate an improvement of up to 5% relative over the standard case where speaker adaptation (during testing and training) is applied after the LDA transform which is trained in a speaker independent manner. We conjecture that performing linear discriminant analysis in a canonical feature space (or speaker normalized space) is more effective than LDA in a speaker independent space because the eigenvectors carve a subspace of maximum intra-speaker phonetic separability whereas in the latter case this subspace is also defined by the inter-speaker variability. Indeed, we show that the more normalization is performed (first VTLN, then SAT), the higher the LDA eigenvalues become.

Unsupervised feature selection and NMF de-noising for robust Speech Emotion Recognition

In Vocal Tract Length Normalization (VTLN) a linear or nonlinear frequency transformation compensates for different vocal tract lengths. Finding good estimates for the speaker specific warp parameters is a critical issue. Despite good results using the Maximum Likelihood criterion to find parameters for a linear warping, there are concerns using this method. We searched for a new criterion that enhances the inter-class separability in addition to optimizing the distribution of each phonetic class. Using such a criterion, Linear Discriminant Analysis determines a linear transformation in a lower dimensional space. For VTLN, we keep the dimension constant and warp the training samples of each speaker such that the Linear Discriminant is optimized. Although that criterion depends on all training samples of all speakers it can iteratively provide speaker specific warp factors. We discuss how this approach can be applied in speech recognition and present first results on two different recognition tasks. 1 Speaker Normalization using VTLN Vocal Tract Length Normalization (VTLN) has proven to decrease the word error rate of a speech recognition system, compared to systems not using such an approach to reduce the variability introduced by different speakers. The main effect addressed here is a shift of the formant frequencies of the speakers caused by their different vocal tract lengths. Two issues have been investigated. The first is how to map one speaker’s spectrum on that of a “standard” or average speaker, depending on a warp parameter which is correlated with the vocal tract length. The other issue is how to find an appropriate warp parameter for each speaker. Most studies assume that the same algorithm is used for training and test, but this is not always necessary. [Acero (1990)] has used a bilinear transform with one speaker dependent parameter. In a first attempt he observed that the algorithm chose a degenerate case where all input frames are transformed into a constant. Therefore, he enforced a constant average warping parameter over all speakers. Modeling the vocal tract as a uniform tube of length L, the formant frequencies are proportional to 1/L. Therefore, some approaches use a linear warp of the frequency scale to normalize speakers. The warp can be performed in the time or spectral domain. In the latter case, a new spectrum is derived by interpolation or by modifying the Mel frequency filter bank. When the warp is applied in the spectral domain, the problem of mismatching frequency ranges occurs. [Wegmann et al (1996)] used a piecewise linear spectral mapping to avoid this problem. They estimated the slope of the transformation function based on a maximum likelihood criterion. [Eide and Gish (1996)] proposed a compromise of different vowel models, namely the uniform tube model and the Helmholtz resonator. They warped the frequency axis f of a speaker according to

This paper presents a new feature extraction algorithm called Power Normalized Cepstral Coefficients (PNCC) that is based on auditory processing. Major new features of PNCC processing include the use of a power-law nonlinearity that replaces the traditional log nonlinearity used in MFCC coefficients, a noise-suppression algorithm based on asymmetric filtering that suppress background excitation, and a module that accomplishes temporal masking. We also propose the use of medium-time power analysis, in which environmental parameters are estimated over a longer duration than is commonly used for speech, as well as frequency smoothing. Experimental results demonstrate that PNCC processing provides substantial improvements in recognition accuracy compared to MFCC and PLP processing for speech in the presence of various types of additive noise and in reverberant environments, with only slightly greater computational cost than conventional MFCC processing, and without degrading the recognition accuracy that is observed while training and testing using clean speech. PNCC processing also provides better recognition accuracy in noisy environments than techniques such as Vector Taylor Series (VTS) and the ETSI Advanced Front End (AFE) while requiring much less computation. We describe an implementation of PNCC using “on-line processing” that does not require future knowledge of the input.

Punjabi is a regional language having variant pronunciations and ranging tones. Therefore developing a robust speech recognition system for such language is today's vital need. Conventional research practices lay stress on using basic extraction techniques like Frequency Cepstral Coefficients (MFCC), Perceptual Linear Prediction (PLP) etc. This papers subject the application of a novel Feature Extraction Technique called Power-Normalized Cepstral Coefficients (PNCC) on connected words in Punjabi Automatic Speech Recognition. This particular technique is an extension to conventional MFCC as MFCC uses mel filter bank and traditional log non linearity on other end the proposed technique uses gammatone filter bank and power law non linearity in acoustic modelling phase. As phoneme sounds (tones) are focused, phone based modelling has been enrolled using 34 phones for training 158 words. The phones are used to break each phoneme word into frames based on the sound produced. Training was done in noise-free environment using 16 speakers and noisy environments with 12 speakers. HTK (Hidden Markov Model toolkit) is used to build acoustic model hatching 8 HMM models for training. Each speaker sheered the speech corpus for number of times to increase the redundancy of the system hence, blooming the recognition rate. Performance analysis was done in noisy-free and noisy environments indulging 3 speakers giving 83.05% accuracy in noisy-free and 71.92% accuracy in noisy environment.

Context. The questions of adapting the convolution neural network classifier use in automatic speaker recognition system of critical use(ASRSCU) are considered. The research object is the individual features of the human speech process.Objective. Development of means for separating individual features from the speaker’s speech signal, increasing their informativeness asa result of the factor analysis, their visual representation for the use of the convolution neural network classifier, and optimizing itsarchitecture for the needs of ASRSCU.Method. Measures are proposed to optimize the speaker recognition procedure of the ASRSCU, for which the optimal way of informativefeatures representation and the method of increasing their informativeness are theoretically justified, the topology and measures for increasingof the speaker recognition process efficiency are justified. In particular, it is justified the use of power normalized cepstral coefficients (PNCC)for the description of phonograms recorded in noisy environment conditions. We propose to use Gabor filters to represent information thatwill be analyzed by a convolution neural network, an optimal method of factor analysis (a sparse main components analyzing method) toreduce of the features vector length while preserving its informativeness, an improved topology of the convolution neural network in whichthe Gabor filters are integrated in to the convolution layer, which allows them to optimize their parameters during the neural network trainingprocess, and in a fully connected layer a deep neural network with a bottleneck layer is used, whose weights after training are uses as inputs forthe GMM/HMM control classifier.Results. Methods of representation and optimization of the speaker’s individual features, methods for their visual presentation andimprovement of the topology of a convolution neural network for making speaker recognition on their basis.Conclusions. The obtained theoretical results have found empirical confirmation. In particular, the stability of an improved convolutionneural network to the noisy input phonograms proved to be higher than the results of an ordinary convolution neural network and a deep neuralnetwork. With an SNR increase up to 10 dB, the GMM/HMM classifier is more efficient than the neural network, which can be explained by the efficiency of the used UBM models, but it is much more resource-intensive. Also, the parameters of the Gabor filter bank frames thatprovide the most variable individual features from the speech signal for speaker recognition are determined empirically.

Speech signal preprocessing is the first and the most important step in the automatic speech recognition process. The preprocessing of speech consists of cleaning the speech signal from ambient and undesirable noises, detecting speech activity, and normalizing the length of the vocal tract. The objective of preprocessing a speech signal is to make the speech recognition systems computationally more efficient through the application of several preprocessing techniques, such as speech pre-emphasis, vocal tract length normalization, voice activity detection, noise removal, framing, and windowing. This paper gives an overview of the fundamentals of speech signal preprocessing techniques, by highlighting the specifics and the requirements of each technique. We also explore all aspects that can improve the results of each technique. We aim that the content of this paper will help researchers improve the quality of their speech recognition systems by identifying appropriate speech preprocessing techniques to use in their experimental settings.

Vocal Tract Length Normalization (VTLN) is used to design vocal tract length normalized Automatic Speech Recognition (ASR) systems. It has led to improvement in the performance of ASR systems by taking into account the physiological differences among speakers. Recently, a number of speech recognition applications are being developed for Indian languages. In this paper, we use state-of-the-art method for VTLN based on maximum likelihood approach. A vowel recognition system has been developed for two low resourced Indian languages, viz., Gujarati and Marathi. Appropriate warping factors have been obtained for all speakers considered for training and testing procedures. An improvement in the performance of vowel recognition is observed as compared to state-of-the-art Mel Frequency Cepstral Coefficients (MFCC).

The automatic recognition of children's speech in acoustically mismatched conditions is a challenging problem on account of large difference in adults' and children's speech. In literature, this challenge is often addressed through concatenation of various feature/model domain adaptation methods like vocal tract length normalization (VTLN), maximum likelihood linear regression (MLLR) and heteroscedastic linear discriminant analysis (HLDA). But a significant gap in the performance of adults and children still remains. This work explores the eigenvoices (EV) based adaptation for addressing the gap in recognition performance of children's speech on adults' speech trained acoustic models. EV is a fast adaptation approach and helps in an effective gender biasing of the acoustic models. On combining EV with VTLN, MLLR and HLDA, under mismatched condition an absolute improvement of about 50% over the unadapted speaker independent system performance is obtained and thus significantly reducing the gap between the performances for adults and children.

A shift-based approach to speaker normalization using non-linear frequency-scaling model

Speech signal is the primary mode of communication amongst humans. It is also used for the communication between human and machine, i.e., human-machine interaction (IoTs), communication with a range of electronic devices, e.g., Internet of Things (IoTs), and even with virtual assistants (VAs) or online search engines, e.g., Google etc. The use of speech signals for the communication over mobile phones, and smart phones based virtual assistants such as Cortana, Siri, Alexa etc. is well known. Usually, this communication involves some communication medium, which is susceptible to noise. The speech or audio signal used for the communication may be adversely affected by the noise that is added unwantedly over the communication channel or medium. It not only degrades the quality of communication and intelligibility of the message or content, but also leads to poor experience. The quality of speech signal may be degraded significantly, due to changes in the characteristics of speech signal, during the process from transmission to reception and the signal conversions involved at both ends. Hence there is a need for algorithms, signal processing methods, acoustic features and classifiers etc. for noise detection and noise reduction during the speech signal communication process. In this paper, a comprehensive review of various speech signal databases, speech signal processing methods used, acoustic features extracted and the classifiers used in different studies is presented. A comprehensive and comparative view of results obtained in different studies, and the applications is also presented. This study should be helpful to all the researchers working in the domains of speech signal processing in noisy environments or degraded speech etc.

A statistically designed Automatic Speech Recognition (ASR) system extracts features from speech signals using feature extraction methods, links the extracted features with the expected phonetics of the hypothesis using acoustic models, uses language model to add prior information about the structure of the target language. For many years, Mel-frequency Cepstral coefficients (MFCC), n-gram, and Hidden Markov Model (HMM) approaches have been used predominantly for feature extraction, language modeling and acoustic modeling, respectively. However, performance degradation of MFCC in noisy conditions and inaccuracy of HMMs while handling large vocabularies have made researchers to propose more efficient methods. The proposed work uses noise robust method Gammatone Frequency Cepstral Coefficients(GFCC) for feature extraction, trigram language modeling, and HMM-Gaussian mixture model (GMM) based acoustic modeling to implement a continuous Hindi language ASR system. Also, it applies Differential Evolution (DE) technique to refine the GFCC features and discriminative techniques to enhance performance of the acoustic model. The performance of the implemented system has been evaluated by using different feature extraction methods, variants of n-gram language modeling techniques and different discriminative techniques in clean as well as noisy conditions. Initially, the results reveal that DE optimized GFCC with HMM-Gaussian Mixture Model (GMM) acoustic modeling performs better than MFCC, PLP and MF-PLP feature extraction methods. Secondly, the experimental results show that the Minimum Phone Error (MPE) outperforms Maximum Mutual Information (MMI) and Maximum Likelihood Estimation (MLE) and trigram based language modeling gives more accurate results than unigram and bigram language modeling. Finally, it has been concluded that the continuous Hindi language ASR system implemented using DE refined GFFC feature extraction method with MPE discriminative training technique and trigram based language modeling gives better accuracy in clean as-well-as noisy environments.

In normal human communication, people face the speaker when listening and usually pay attention to the speaker' face. Therefore, in robot audition, the recognition of the front talker is critical for smooth interactions. This paper presents an enhanced speech detection method for a humanoid robot that can separate and recognize speech signals originating from the front even in noisy home environments. The robot audition system consists of a new type of voice activity detection (VAD) based on the complex spectrum circle centroid (CSCC) method and a maximum signal-to-noise (Max-SNR) beamformer. This VAD based on CSCC can classify speech signals that are retrieved at the frontal region of two microphones embedded on the robot. The system works in real-time without needing training filter coefficients given in advance even in a noisy environment (SNR ≫ 0 dB). It can cope with speech noise generated from televisions and audio devices that does not originate from the center. Experiments using a humanoid robot, SIG2, with two microphones showed that our system enhanced extracted target speech signals more than 12 dB (SNR) and the success rate of automatic speech recognition for Japanese words was increased about 17 points.

The communication mobile devices increasingly require automatic distributed speech recognition (ADSR) to be robust to the full range of real-environments noise and other acoustic distorting conditions. This paper aims to establish a solid framework integrating a noise-robust frontend in ADSR, which is lacking at present. The European Telecommunications Standards Institute advanced frontend (ETSI-AFE) is commonly referred method which performs very well in noisy environments, but poorly in noise-free conditions, and it is primarily suitable for dealing with stationary noise. We are intended to fill this gap and evaluate the development of an additional feature set for Mel frequency based ETSI-AFE, such as the power-normalized cepstral coefficients (PNCC). Thereafter, the stream weights of the resulting multi-stream hidden Markov models are optimized automatically by deploying a novel approach based on a discriminative model combination (DMC). This system DMC-MFCC frontend achieves significant performance improvements over related systems under a wide range of signal-to-noise ratios.KeywordsDistributed speech recognitionAdvanced frontendDiscriminative model combinationMFCCPNCC