Speaker Recognition using Gaussian Mixture Models

In the biometrics community, face and speaker recognition are mature fields in which several systems have been proposed over the past twenty years. While existing systems perform well under controlled recording conditions, mismatch caused by the use of different sensors or a lack of cooperation from the subject still significantly affects performance, especially in challenging scenarios such as in forensics. Furthermore, existing methods suffer from scalability issues, which prevents them from taking advantage of increasingly large amounts of training data. This is otherwise a promising approach to improve accuracy in such challenging scenarios. In this thesis we address these problems of mismatch and complexity by developing scalable probabilistic models that we apply to face, speaker and bimodal recognition. Our contributions are four-fold. First, we propose a unified framework for session variability modeling techniques based on Gaussian mixture models (GMM), that encompasses inter-session variability (ISV) modeling, joint factor analysis (JFA) and total variability (TV) modeling. Second, we propose a novel exact and scalable formulation of probabilistic linear discriminant analysis (PLDA), which is a probabilistic and generative framework that models between-class and within-class variations. This formulation solves a major scalability issue, by improving both the time complexity of the training procedure from cubic to linear with respect to the number of samples per class, and the complexity of the scoring procedure. Furthermore, the implementations of all the proposed techniques are integrated into a novel collaborative open source software library called Bob 1 that enforces fair evaluations and encourages reproducible research. Fourth and finally, large-scale experiments are conducted with all of the above machine learning algorithms on several databases such as FRGC for face recognition, NIST SRE12 for speaker recognition and MOBIO for bimodal recognition, showing competitive performances.

Speaker and language recognition in noisy and degraded channel conditions continue to be a challenging problem mainly due to the mismatch between clean training and noisy test conditions. In the presence of noise, the most reliable portions of the signal are the high energy regions which can be used for robust feature extraction. In this paper, we propose a front end processing scheme based on autoregressive (AR) models that represent the high energy regions with good accuracy followed by a modulation filtering process. The AR model of the spectrogram is derived using two separable time and frequency AR transforms. The first AR model (temporal AR model) of the sub-band Hilbert envelopes is derived using frequency domain linear prediction (FDLP). This is followed by a spectral AR model applied on the FDLP envelopes. The output 2-D AR model represents a low-pass modulation filtered spectrogram of the speech signal. The band-pass modulation filtered spectrograms can further be derived by dividing two AR models with different model orders (cut-off frequencies). The modulation filtered spectrograms are converted to cepstral coefficients and are used for a speaker recognition task in noisy and reverberant conditions. Various speaker recognition experiments are performed with clean and noisy versions of the NIST-2010 speaker recognition evaluation (SRE) database using the state-of-the-art speaker recognition system. In these experiments, the proposed front-end analysis provides substantial improvements (relative improvements of up to 25%) compared to baseline techniques. Furthermore, we also illustrate the generalizability of the proposed methods using language identification (LID) experiments on highly degraded high-frequency (HF) radio channels and speech recognition experiments on noisy data.

Due to the rapid advances in algorithms, VLSI design and computer technology, security systems based on speaker recognition are on the verge of commercial success. In this thesis, an improved strategy for Text Tndependent Automatic Speaker Verification (TI-ASV) system based on Malayalam has been proposed and results are observed. The system performs on Gaussian Mixture Model (GMM) technique with cepstral based features. Different speech pre-processing techniques like pre-emphasis filtering, frame blocking and windowing have been used to process the speech utterances. MFCC, ΔMFCC and ΔΔMFCC have been used to extract the features. Speaker identification is performed using Continuous Hidden Markov Model. The performance is analyzed in terms of Percentage Correctness (PC) and accuracy. An application with Graphical User Interface (GUI) is also developed for security purposes using the system. The system is developed using the framework of Hidden Markov Model Tool Kit (HTK).

The Gaussian mixture models (GMM) represent an efficient model that was broadly used in most of speaker recognition applications. This study introduces a novel method for speaker verification task. We propose a reduced feature vector employing new information detected from the speaker's voice for performing text-independent speaker verification applications using GMM. We use the power spectrum density of the speech signal to improve the system's performance. Speaker verification experiments were evaluated with the TIMIT dataset. The suggested system performance is evaluated against the baseline systems. The decrease in the error rate is well observed and the results have demonstrated the effectiveness of the new approach which avoids the use of more complex algorithms or the combination of different approaches.

Investigation of the effect of data duration and speaker gender on text-independent speaker recognition

Automatic speaker recognition may achieve remarkable performance in matched training and test conditions. Conversely, results drop significantly in incompatible noisy conditions. Furthermore, feature extraction significantly affects performance. Mel-frequency Cepstral coefficients MFCCs are most commonly used in this field of study. The literature has reported that the conditions for training and testing are highly correlated. Taken together, these facts support strong recommendations for using MFCC features in similar environmental conditions (train/test) for speaker recognition. However, with noise and reverberation present, MFCC performance is not reliable. To address this, we propose a new feature 'Entrocy' for accurate and robust speaker recognition, which we mainly employ to support MFCC coefficients in noisy environments. Entrocy is the Fourier Transform of the Entropy, a measure of the fluctuation of the information in sound segments over time. Entrocy features are combined with MFCCs to generate a composite feature set which is tested using the Gaussian Mixture Model (GMM) speaker recognition method. The proposed method shows improved recognition accuracy over a range of signal-to- noise ratios.

Relevance factor of maximum a posteriori adaptation for GMM–NAP–SVM in speaker and language recognition

This study aims to develop an elderly care system for improving the interpersonal relationship of the elderly with mild cognitive impairment (MCI) by employing the speaker recognition technique. The speaker recognition units based on the Gaussian Mixture Model (GMM) and Gaussian Mixture Model-Universal Background Model (GMM-UBM) are implemented to identify the visitor via individual input utterance. Experimental results indicate that the speaker recognition unit based on GMM-UBM achieves the best performance.

Speaker identification identifies the speaker among a set of users by matching against a set of voiceprints. In speaker identification, the identification time depends on the number of feature vectors, their dimensionality and the number of speakers. In this paper, text independent speaker identification model is developed by taking in MFCCs with VQ to obtain pressed feature vectors without losing much information, and the numbers of speakers are reduced in the test by gender detection algorithm. Gaussian Mixture Model (GMM) is used a modeling technique. Results show that proposed approach always yields better improvements in accuracy and brings almost 50% reduces in time processing.

Residual networks for text-independent speaker identification: Unleashing the power of residual learning

Classification of multi-variate time series data of varying length finds applications in various domains of science and technology. There are two paradigms for modeling multi-variate varying length time series, namely, modeling the sequences of feature vectors and modeling the sets of feature vectors in the time series. In tasks such as text independent speaker recognition, audio clip classification and speech emotion recognition, modeling temporal dynamics is not critical and there may not be any underlying constraint in the time series. Gaussian mixture models (GMM) are commonly used for these tasks. In this paper, we propose a method based on descriptive statistical features for multi-variate varying length time series classification. The proposed method reduces the dimensionality of representation significantly and is less sensitive to missing samples. The proposed method is applied on speech emotion recognition and audio clip classification. The performance is compared with that of the GMMs based approaches that use maximum likelihood method and variational Bayes method for parameter estimation, and two approaches that combine GMMs and SVMs, namely, score vector based approach and segment modeling based approach. The proposed method is shown to give a better performance compared to all other methods.

Recent experiments in speech synthesis have shown, that by an appropriate eigenvector analysis, a set of orthogonal parameters can be obtained that is essentially independent of all linguistic information across an analyzed utterance but highly indicative of the identity of the speaker. The orthogonal parameters are formed by a linear transformation of the linear prediction parameters, and can achieve their recognition potential without the need of any time normalization procedure. The speaker discrimination potential of the linear prediction orthogonal parameters were formally tested in both a speaker recognition and a speaker verification experiment. The speech data for these experiments consisted of six repetitions of the same sentence spoken by 21 male speakers on six separate occasions. For both recognition and verification, the identification accuracy of the orthogonal parameters exceeded 99%. In a separate text-independent speaker recognition experiment, an accuracy of 94% was achieved.

Speech applications, which operate a system by voice commands, facilitate web access for disabled and visually impaired users. Human-computer interactions, such as speaking and listening to web applications, provide options for developing a multimodal interaction tool in the accessible design of an intelligent web. Speaker identification and verification are essential functionalities for intelligent web programs with speech applications. This paper proposes an enhanced Gaussian mixture model (GMM) method by incorporating the information derived from the support vector machine (SVM), called EGMM-SVM, for web-based applications with speaker recognition. The EGMM-SVM improves the accuracy of the estimated likelihood scores between the speech frame and the GMM. In EGMM-SVM, SVM plays a crucial role in transmitting the quality information of the utterances from a test speaker, through the GMM when performing GMM likelihood calculations. The experimental results show that speaker recognition by using the developed EGMM-SVM with an accurate operation mechanism for Gaussian distribution derivations yields a higher recognition rate than does a conventional GMM without any considerations on the quality of test speech utterances.

The best performing systems in the area of automatic speaker recognition have focused on using short-term, low-level acoustic information, such as cepstral features. Recently, various works have demonstrated that high-level features convey more speaker information and can be added to the low-level features in order to increase the robustness of the system. This paper describes a text-independent speaker recognition system exploiting high-level information provided by ALISP (Automatic Language Independent Speech Processing), a data-driven segmentation. This system, denoted here as ALISP n-gram system, captures the speaker specific information only by analyzing sequences of ALISP units. The ALISP n-gram system was fused with an acoustic ALISP-based Gaussian Mixture Models (GMM) system exploiting the speaker discriminating properties of individual speech classes. The resulting fused system reduced the error rate over the individual systems on the NIST 2004 Speaker Recognition Evaluation data.KeywordsGaussian Mixture ModelBackground ModelFalse Alarm ProbabilitySpeech DataSpeaker RecognitionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

To make the supervector modeling of speech utterance more effective and accurate, this paper proposes a new duration weighted Gaussian Mixture Model (GMM) supervector modeling method for robust speaker recognition. At the beginning, this method adapts the acoustic features of speech utterance to GMM from a common basic Universal Background Model (UBM) with Maximum A Posterior (MAP) criterion and then models GMM supervector by bounding the Kullback-Leibler (KL) divergence measure. In addition, a duration weight supervector is modeled for using duration information of speech utterances. Furthermore, this paper presents a method of how to effectively apply them together during training and classification. Experimental results on American National Institute of Standards and Technology Speaker Recognition Evaluation (NIST SRE) 2008 dataset demonstrate that the proposed method outperforms the traditional GMM supervector modeling with relative 16% and 10% improvements of Equal Error Rate (EER) and Minimum Detection Cost Function (MinDCF), respectively.