Contextual invariant-integration features for improved speaker-independent speech recognition

This paper proposes a novel feature-space VTLN (vocal tract length normalization) method that models frequency warping as a linear interpolation of contiguous Mel filter-bank energies. The presented technique aims to reduce the distortion in the Mel filter-bank energy estimation due to the harmonic composition of voiced speech intervals and DFT (discrete Fourier transform) sampling when the central frequency of band-pass filters is shifted. This paper also proposes an analytical maximum likelihood (ML) method to estimate the optimal warping factor in the cepstral space. The presented interpolated filter-bank energy-based VTLN leads to relative reductions in WER (word error rate) as high as 11.2% and 7.6% when compared with the baseline system and standard VTLN, respectively, in a medium-vocabulary continuous speech recognition task. Also, the proposed VTLN scheme can provide significant reductions in WER when compared with state-of-the-art VTLN methods based on linear transforms in the cepstral feature-space. The warping factor estimated with the proposed VTLN approach shows more dependence on the speaker and more independence of the acoustic-phonetic content than the warping factor resulting from standard and state-of-the-art VTLN methods. Finally, the analytical ML-based optimization scheme presented here achieves almost the same reductions in WER as the ML grid search version of the technique with a computational load 20 times lower.

This paper shows the impact of speaker normalization techniques, such as vocal tract length normalization (VTLN) and speaker-adaptive training (SAT), prior to discriminant feature space transforms, such as LDA (linear discriminant analysis). We demonstrate that removing the inter-speaker variability by using speaker compensation methods results in improved discrimination as measured by the LDA eigenvalues and also in improved classification accuracy (as measured by the word error rate). Experimental results on the SPINE (speech in noisy environments) database indicate an improvement of up to 5% relative over the standard case where speaker adaptation (during testing and training) is applied after the LDA transform which is trained in a speaker independent manner. We conjecture that performing linear discriminant analysis in a canonical feature space (or speaker normalized space) is more effective than LDA in a speaker independent space because the eigenvectors carve a subspace of maximum intra-speaker phonetic separability whereas in the latter case this subspace is also defined by the inter-speaker variability. Indeed, we show that the more normalization is performed (first VTLN, then SAT), the higher the LDA eigenvalues become.

In Vocal Tract Length Normalization (VTLN) a linear or nonlinear frequency transformation compensates for different vocal tract lengths. Finding good estimates for the speaker specific warp parameters is a critical issue. Despite good results using the Maximum Likelihood criterion to find parameters for a linear warping, there are concerns using this method. We searched for a new criterion that enhances the inter-class separability in addition to optimizing the distribution of each phonetic class. Using such a criterion, Linear Discriminant Analysis determines a linear transformation in a lower dimensional space. For VTLN, we keep the dimension constant and warp the training samples of each speaker such that the Linear Discriminant is optimized. Although that criterion depends on all training samples of all speakers it can iteratively provide speaker specific warp factors. We discuss how this approach can be applied in speech recognition and present first results on two different recognition tasks. 1 Speaker Normalization using VTLN Vocal Tract Length Normalization (VTLN) has proven to decrease the word error rate of a speech recognition system, compared to systems not using such an approach to reduce the variability introduced by different speakers. The main effect addressed here is a shift of the formant frequencies of the speakers caused by their different vocal tract lengths. Two issues have been investigated. The first is how to map one speaker’s spectrum on that of a “standard” or average speaker, depending on a warp parameter which is correlated with the vocal tract length. The other issue is how to find an appropriate warp parameter for each speaker. Most studies assume that the same algorithm is used for training and test, but this is not always necessary. [Acero (1990)] has used a bilinear transform with one speaker dependent parameter. In a first attempt he observed that the algorithm chose a degenerate case where all input frames are transformed into a constant. Therefore, he enforced a constant average warping parameter over all speakers. Modeling the vocal tract as a uniform tube of length L, the formant frequencies are proportional to 1/L. Therefore, some approaches use a linear warp of the frequency scale to normalize speakers. The warp can be performed in the time or spectral domain. In the latter case, a new spectrum is derived by interpolation or by modifying the Mel frequency filter bank. When the warp is applied in the spectral domain, the problem of mismatching frequency ranges occurs. [Wegmann et al (1996)] used a piecewise linear spectral mapping to avoid this problem. They estimated the slope of the transformation function based on a maximum likelihood criterion. [Eide and Gish (1996)] proposed a compromise of different vowel models, namely the uniform tube model and the Helmholtz resonator. They warped the frequency axis f of a speaker according to

This paper describes the work done in improving the performance of Tamil speech recognition system by using Time Scale Modification (TSM) and Vocal Tract Length Normalization (VTLN) techniques. The speech recognition system for Tamil language was developed using a new approach of text independent speech segmentation, with a phoneme based language model for recognition. There is degradation in the performance of speech recognition due to variations in the speaking rate and vocal tract shape among different speakers. In order to improve the performance of speech recognition system, both TSM and VTLN normalization techniques were used in this work. The TSM was implemented using the Phase vocoder approach and the VTLN was implemented using speaker specific bark/mel scale in bark/mel domain. The performance of Tamil speech recognition system was improved by performing both TSM and VTLN normalization techniques.

Children’s Speech Recognition (CSR) is a challenging task due to the high variability in children’s speech patterns and limited amount of available annotated children’s speech data. We aim to improve CSR in the often-occurring scenario that no children’s speech data is available for training the Automatic Speech Recognition (ASR) systems. Traditionally, Vocal Tract Length Normalization (VTLN) has been widely used in hybrid ASR systems to address acoustic mismatch and variability in children’s speech when training models on adults’ speech. Meanwhile, End-to-End (E2E) systems often use data augmentation methods to create child-like speech from adults’ speech. For adult speech-trained ASRs, we investigate the effectiveness of augmentation methods; speed perturbations and spectral augmentation, along with VTLN, in an E2E framework for the CSR task, comparing these across Dutch, German, and Mandarin. We applied VTLN at different stages (training/test) of the ASR and conducted age and gender analyses. Our experiments showed highly similar patterns across the languages: Speed Perturbations and Spectral Augmentation yield significant performance improvements, while VTLN provided further improvements while maintaining recognition performance on adults’ speech (depending on when it is applied). Additionally, VTLN showed performance improvement for both male and female speakers and was particularly effective for younger children.

The performance of automatic speech recognition systems for children’s speech is known to suffer from the large variation and mismatch in the acoustic and linguistic attributes between children’s and adults’ speech. One of the various identified sources of mismatch is the difference in formant frequencies between adults and children. In this paper, we propose a formant modification method to mitigate differences between adults’ and children’s speech and to improve the performance of ASR for children. The explored technique gives a relative 27% improvement in system performance compared to a hybrid DNN-HMM baseline. We also compare the system performance with related speaker adaptation methods like vocal tract length normalization (VTLN) and speaking rate adaptation (SRA) and find that the proposed method gives improvements over them, as well. Combining the proposed method with VTLN and SRA results in a further reduction of WER. We also found that the proposed method performs well even for noisy speech.

Vocal Tract Length Normalization (VTLN) is used to design vocal tract length normalized Automatic Speech Recognition (ASR) systems. It has led to improvement in the performance of ASR systems by taking into account the physiological differences among speakers. Recently, a number of speech recognition applications are being developed for Indian languages. In this paper, we use state-of-the-art method for VTLN based on maximum likelihood approach. A vowel recognition system has been developed for two low resourced Indian languages, viz., Gujarati and Marathi. Appropriate warping factors have been obtained for all speakers considered for training and testing procedures. An improvement in the performance of vowel recognition is observed as compared to state-of-the-art Mel Frequency Cepstral Coefficients (MFCC).

The automatic recognition of children's speech in acoustically mismatched conditions is a challenging problem on account of large difference in adults' and children's speech. In literature, this challenge is often addressed through concatenation of various feature/model domain adaptation methods like vocal tract length normalization (VTLN), maximum likelihood linear regression (MLLR) and heteroscedastic linear discriminant analysis (HLDA). But a significant gap in the performance of adults and children still remains. This work explores the eigenvoices (EV) based adaptation for addressing the gap in recognition performance of children's speech on adults' speech trained acoustic models. EV is a fast adaptation approach and helps in an effective gender biasing of the acoustic models. On combining EV with VTLN, MLLR and HLDA, under mismatched condition an absolute improvement of about 50% over the unadapted speaker independent system performance is obtained and thus significantly reducing the gap between the performances for adults and children.

Punjabi is a regional language having variant pronunciations and ranging tones. Therefore developing a robust speech recognition system for such language is today's vital need. Conventional research practices lay stress on using basic extraction techniques like Frequency Cepstral Coefficients (MFCC), Perceptual Linear Prediction (PLP) etc. This papers subject the application of a novel Feature Extraction Technique called Power-Normalized Cepstral Coefficients (PNCC) on connected words in Punjabi Automatic Speech Recognition. This particular technique is an extension to conventional MFCC as MFCC uses mel filter bank and traditional log non linearity on other end the proposed technique uses gammatone filter bank and power law non linearity in acoustic modelling phase. As phoneme sounds (tones) are focused, phone based modelling has been enrolled using 34 phones for training 158 words. The phones are used to break each phoneme word into frames based on the sound produced. Training was done in noise-free environment using 16 speakers and noisy environments with 12 speakers. HTK (Hidden Markov Model toolkit) is used to build acoustic model hatching 8 HMM models for training. Each speaker sheered the speech corpus for number of times to increase the redundancy of the system hence, blooming the recognition rate. Performance analysis was done in noisy-free and noisy environments indulging 3 speakers giving 83.05% accuracy in noisy-free and 71.92% accuracy in noisy environment.

Aiming at the problem of inaccurate event location in noisy environment by existing acoustic even detection technology, this paper presents an acoustic event detection algorithm based on two-stage judgement. Firstly, the acoustic events existing in the audio signal are located by the two-stage judgement detection method, both of the distance of the Mel Frequency Cepstral Coefficients (MFCC) and the short-time energy between each audio signal frame and the noise average are calculated, respectively. The MFCC distance in the frequency domain which can produce fine but incomplete results is the first judgement; the energy distance in the time domain is the second judgement, which is used to supplement the first judgment. Studies have shown that the Gammatone filter bank is biologically closer to the human ear structure than the Mel filter bank. The Gammatone Frequency Cepstral Coefficients (GFCC) of the detected acoustic events were then extracted. The detected acoustic events are classified by the Gaussian Mixture Model (GMM). By analyzing the experimental results, the algorithm can solve the problem that the sound feature information is insufficient and the noise segment boundary is not clear. This system is more suitable for the situation where a variety of acoustic events should be analyzed.

Noise robust in-domain children speech enhancement for automatic Punjabi recognition system under mismatched conditions

This work investigates the use of subglottal resonances (SGRs) for speaker normalization in noisy environments. Based on our previous work, a noise-robust algorithm is developed for estimating the first three SGRs from speech signals; it achieves robustness by factoring the short-term (or local) signal-to-noise ratio into the estimation process. The SGR estimates provided by this algorithm are refined by applying maximum-likelihood (ML) corrections, and are used in a non-linear frequency-warping technique that we recently developed. This SGR-based normalization (SN) scheme is evaluated on the AURORA-4 database in clean and noisy conditions. Using power-normalized cepstral coefficients (PNCCs) as front-end features, SN reduces the average word error rate by 8.7% relative to ML-based vocal-tract length normalization (VTLN). A fast version of SN (without ML corrections of SGR estimates) is also found to outperform VTLN (by 5.9% relative); it is computationally less complex than VTLN and hence a potential alternative for real-time applications.

Speech signal preprocessing is the first and the most important step in the automatic speech recognition process. The preprocessing of speech consists of cleaning the speech signal from ambient and undesirable noises, detecting speech activity, and normalizing the length of the vocal tract. The objective of preprocessing a speech signal is to make the speech recognition systems computationally more efficient through the application of several preprocessing techniques, such as speech pre-emphasis, vocal tract length normalization, voice activity detection, noise removal, framing, and windowing. This paper gives an overview of the fundamentals of speech signal preprocessing techniques, by highlighting the specifics and the requirements of each technique. We also explore all aspects that can improve the results of each technique. We aim that the content of this paper will help researchers improve the quality of their speech recognition systems by identifying appropriate speech preprocessing techniques to use in their experimental settings.

Speaker-normalization and -adaptation methods are essential components of state-of-the-art speech recognition systems nowadays. Recently, so-called invariant integration features were presented which are motivated by the theory of invariants. While it was shown that the integration features outperform MFCCs when used with a basic monophone recognition system, it was left open, if their benefits still can be observed when a more sophisticated recognition system with speaker-normalization and/or speaker-adaptation components is used. This work investigates the combination of the integration features with standard speaker-normalization and -adaptation methods. We show that the integration features benefit from adaptation methods and significantly outperform MFCCs in matching, as well as in mismatching training-test conditions. Index Terms: Speaker independency, invariant integration, speaker normalization, speaker adaptation

Vocal tract length normalization (VTLN) is a process of reducing the effects of VTL differences among speakers. In this paper, frequency warping based on Bark scale is presented to obtain VTL normalized features and Bark scale based features (BSBF) have been proposed. BSBFs are found to improve the accuracy of vowel recognition task by 1.05 % over MFCCs. A feature-level fusion of BSBFs with MFCCs performs better than MFCCs alone under matched and mismatched conditions in phoneme recognition.