New Missing Features Mask Estimation Method for Speaker Recognition in Noisy Environments

In order to handle speech signals corrupted by noise in speaker verification and provide robustness to systems, this paper evaluates the use of missing feature (MF) approach with a novel combination of techniques. A mask estimation based on spectral subtraction is used to determine the reliability of spectral components in a speech signal corrupted by noise. A cluster based reconstruction technique is used to remake the damaged spectrum. The verification performance was evaluated through a speaker verification experiment with signals corrupted by white noise under different signal to noise ratios. The results were promising since they reflected a relevant increase of speaker verification performance, applyingMF approach with this combination of techniques.

The mismatch between the training conditions and the test conditions severely degrades the performance of speaker verification. Aiming at solving this problem, this paper presents a method which uses a combination of improved nonnegative matrix factorization (IMNMF) and mel frequency cepstral coefficients (MFCC) with feature warping for improving identity-vector (i-vector) speaker verification performance in noisy environment. Unlike the traditional nonnegative matrix factorization (NMF), IMNMF uses extra free basis vectors to capture the features which are not included in training data, and linear constraints on dictionary atoms. Feature warping is used to remove channel noises. Therefore, the proposed method can reduce distortion of reconstructed speech while enhancing the recovered speech quality. The performance of i-vector speaker verification is evaluated by using the short utterance database and the NOISEX-92 database. The experiment results indicate that the score level fusion of feature-warped IMNMF-MFCC and feature-warped MFCC is superior to the baseline system at the equal error rate under the majority of signal-to-noise ratios (SNRs).

A new large margin nearest neighborhood metric learning (LMNN) method for i-vector based speaker verification is proposed in this paper. In general, a verification decision is taken by computing the cosine distance between the i-vectors of the test utterance and the claimed identity. LMNN metric is learned from the examples and can be viewed as a linear transformation of the input i-vector space of the training and test utterance. In this work, the metric is learned with the objective of reducing the distance between the i-vectors of same class of speaker, while impostors are separated by a large margin. The metric learned in this manner leads to a better speaker verification performance. Speaker verification experiments are then conducted on the NIST 2008 and YOHO speaker verification databases. Experimental results indicate a reasonable improvement in performance, when compared to i-vector based speaker verification methods which use conventional cosine scoring.

In this paper, we present a Mixture Linear Prediction based approach for robust Gammatone Cepstral Coefficients extraction (MLPGCCs). The proposed method provides performance improvement of Automatic Speaker Verification (ASV) using i-vector and Gaussian Probabilistic Linear Discriminant Analysis GPLDA modeling under transmission channel noise. The performance of the extracted MLPGCCs was evaluated using the NIST 2008 database where a single channel microphone recorded conversational speech. The system is analyzed in the presence of different channel transmission noises such as Additive White Gaussian (AWGN) and Rayleigh fading at various Signals to Noise Ratio (SNR) levels. The evaluation results show that the MLPGCCs features are a promising way for the ASV task. Indeed, the speaker verification performance using the MLPGCCs proposed features is significantly improved compared to the conventional Gammatone Frequency Cepstral Coefficients (GFCCs) and Mel Frequency Cepstral Coefficients (MFCCs) features. For speech signals corrupted with AWGN noise at SNRs ranging from (-5 dB to 15 dB), we obtain a significant reduction of the Equal Error Rate (EER) ranging from 9.41% to 6.65% and 3.72% to 1.50%, compared with conventional MFCCs and GFCCs features respectively. In addition, when the test speech signals are corrupted with Rayleigh fading channel we achieve an EER reduction ranging from 23.63% to 7.8% and from 10.88% to 6.8% compared with conventional MFCCs and GFCCs, respectively. We also found that the combination of GFCCs and MLPGCCs gives the highest performance of speaker verification system. The best performance combination achieved is around EER from 0.43% to 0.59% and 1.92% to 3.88%.

This is the author’s version of a work that was submitted/accepted for pub-lication in the following source:

The popularity and application of smart home devices have made far-field speaker verification an urgent need. However, speaker verification performance is unsatisfactory under far-field environments despite its significant improvements enabled by deep neural networks (DNN). In this paper, we summarize our previous work and propose multiple training strategies and models for multi-channel far-field speaker verification with different in-domain data availability scenarios. The experiments are conducted on the FFSVC20 dataset, and we proposed the cross-device and cross-domain trials. We focus on single-channel and multi-channel speaker verification training based on the dataset. For single-channel speaker verification, considering the size of training data and availability of labels, we introduce three training scenarios and given our proposed training methods, including 1) given zero out-of-domain data and few in-domain labeled data; 2) given large-scale out-of-domain labeled data and few in-domain labeled data; 3) given large-scale out-of-domain labeled data and few in-domain unlabeled data. To this end, we propose a meta-learning approach, refined transfer learning methods, and semi-supervised learning for three scenarios, respectively. For multi-channel speaker verification, we first introduce two types of 3 dimension convolution (3D Conv) residual network (ResNet) models proposed in our previous works, including fully 3D ResNet and incorporating 3D Conv with 2D Conv ResNet (3D2D-ResNet). In this paper, we propose channel-wise 3D squeeze-and-excitation ResNet (C3DSE-ResNet) and spatial-wise 3D SE ResNet (S3DSE-ResNet) to further explore the channel dependencies and improve the 3D ConvNet performance. The results show that the proposed strategies and models can significantly boost performance under the far-field scenario.

Language mismatching is considered one of the biggest challenges in achieving adequate speaker verification. The number of bilingual speakers worldwide is increasing, making speaker verification for speech technology more challenging. The main objective of this study is to examine the effect of language mismatch between training and test conditions on the performance of the speaker verification model, with a specific focus on children's speech by utilizing pre-trained and fine-tuned deep speaker embedding models and by investigating two child speech datasets (Samromur and kidsTALC). This work utilises two time-delay neural networks (TDNN): X-Vector and Emphasized Channel Attention, Propagation and Aggregation (ECAPA) to extract the embedding features of speech samples. For evaluation of the performance of speaker verification, we used the likelihood-ratio framework by using the likelihood-ratio score calculation method based on children’s voices and employing measures such as log-likelihood ratio cost (Cllr) and equal error rate (EER). The experimental results indicate that the language variety between training and testing utterances significantly degrades speaker verification performance compared to multi-language training; however, fine-tuning still performs better than pretrained models by 11.5% in the best performance of ECAPA-TDNN.

In recent years, speaker verification technologies have received an extensive amount of attention. Designing and developing machines that could communicate with humans are believed to be one of the primary motivations behind such developments. Speaker verification technologies are applied to numerous fields such as security, Biometrics, and forensics. In this paper, the authors study the effects of different languages on the performance of the automatic speaker verification (ASV) system. The MirasVoice speech corpus (MVSC), a bilingual English and Farsi speech corpus, is used in this study. This study collects results from both an I-vector based ASV system and a GMM-UBM based ASV system. The experimental results show that a mismatch between the enrolled data used for training and verification data can lead to a significant decrease in the overall system efficiency. This study shows that it is best to use an i-vector based framework with data from the English language used in the enrollment phase to improve the robustness of the ASV systems. The achieved results in this study indicate that this can narrow the degradation gap caused by the language mismatch.

This work is dedicated to introducing, executing, and assessing a three-stage speaker verification framework to enhance the degraded speaker verification performance in emotional talking environments. Our framework is comprised of three cascaded stages: gender identification stage followed by an emotion identification stage followed by a speaker verification stage. The proposed framework has been assessed on two distinct and independent emotional speech datasets: our collected dataset and Emotional Prosody Speech and Transcripts dataset. Our results demonstrate that speaker verification based on both gender cues and emotion cues is superior to each of speaker verification based on gender cues only, emotion cues only, and neither gender cues nor emotion cues. The achieved average speaker verification performance based on the suggested methodology is very similar to that attained in subjective assessment by human listeners.

A method is described for estimating telephone handset nonlinearity by matching the spectral magnitude of the distorted signal to the output of a nonlinear channel model, driven by an undistorted reference. This "magnitude only" representation allows the model to directly match unwanted speech formants that arise over nonlinear channels and that are a potential source of degradation in speaker and speech recognition algorithms. As such, the method is particularly suited to algorithms that use only spectral magnitude information. The distortion model consists of a memoryless nonlinearity sandwiched between two finite-length linear filters. Nonlinearities considered include arbitrary finite-order polynomials and parametric sigmoidal functionals derived from a carbon-button handset model. Minimization of a mean-squared spectral magnitude distance with respect to model parameters relies on iterative estimation via a gradient descent technique. Initial work has demonstrated the importance of addressing handset nonlinearity, in addition to linear distortion, in speaker recognition over telephone channels. A nonlinear handset "mapping," applied to training or testing data to reduce mismatch between different types of handset microphone outputs, improves speaker verification performance relative to linear compensation only. Finally, a method is proposed to merge the mapper strategy with a method of likelihood score normalization (hnorm) for further mismatch reduction and speaker verification performance improvement.

ABSTRACT: Speech is one of the most natural and efficient forms of communication among humans, and it is becoming increasingly integral to human-computer interaction in modern applications such as virtual assistants, smart devices, and automated customer service systems. Among the various technologies that leverage speech, speaker recognition stands out as a biometric method that identifies or verifies individuals based on the unique characteristics of their voice. This approach has gained significant traction due to its non-intrusive nature, ease of integration, and applicability across various domains such as security, forensics, and personalised user experiences. In this paper, we introduce a novel speaker recognition system specifically designed to identify speakers based on utterances in the Marathi language, a linguistically rich and widely spoken regional language in India. The system utilises Mel-Frequency Cepstral Coefficients (MFCCS) to extract distinguishing vocal features that closely mimic the human auditory perception. MFCCS are particularly effective in capturing speech’s phonetic and acoustic properties, making them a preferred choice in speech and speaker recognition tasks. Vector quantisation (VQ) is applied to optimise the feature set and reduce computational complexity. VQ compresses the high-dimensional MFCC feature vectors into representative clusters without significantly compromising accuracy, thereby enhancing system efficiency. These features are then processed using hidden Markov Models (HMMS), which are adept at modelling temporal sequences and dynamic variations in speech. HMMS offer a statistical framework that effectively captures the sequential nature of speech patterns, leading to more reliable speaker modelling and recognition.

Voice biometrics (e.g., Speaker Verification) is a critical type of biometrics based on human voice characteristics and is known for security and user-friendliness. It has been widely applied in worldwide applications, such as voice assistants and online banking. However, a concern is raised rapidly about the demographic fairness that different subgroups may have different speaker verification performance due to the inherent voice characteristics. And little work done investigates this concern. A diverse group of 300 speakers by race and gender is recruited for exploration. After running some speaker verification evaluations, three conclusions were reached. Firstly, the Latinx are performed the worst among the four major races in the US (White, Black, Latinx, and Asian) in speaker verification. Secondly, that gender shows little difference in performance between men and women. Thirdly, that high entropy voices performed better than low entropy voices in speaker verification performance.

In this paper, a novel architecture for speaker recognition is proposed by cascading speech enhancement and speaker processing. Its aim is to improve speaker recognition performance when speech signals are corrupted by noise. Instead of individually processing speech enhancement and speaker recognition, the two modules are integrated into one framework by a joint optimisation using deep neural networks. Furthermore, to increase robustness against noise, a multi-stage attention mechanism is employed to highlight the speaker related features learned from context information in time and frequency domain. To evaluate speaker identification and verification performance of the proposed approach, we test it on the dataset of VoxCeleb1, one of mostly used benchmark datasets. Moreover, the robustness of our proposed approach is also tested on VoxCeleb1 data when being corrupted by three types of interferences, general noise, music, and babble, at different signal-to-noise ratio (SNR) levels. The obtained results show that the proposed approach using speech enhancement and multi-stage attention models outperforms two strong baselines not using them in most acoustic conditions in our experiments.

Over recent years, i-vector-based framework has been proven to provide state-of-the-art performance in speaker verification. Each utterance is projected onto a total factor space and is represented by a low-dimensional feature vector. Channel compensation techniques are carried out in this low-dimensional feature space. Most of the compensation techniques take the sets of extracted i-vectors as input. By constructing between-class covariance and within-class covariance, we attempt to minimize the between-class variance mainly caused by channel effect and to maximize the variance between speakers. In the real-world application, enrollment and test data from each user (or speaker) are always scarce. Although it is widely thought that session variability is mostly caused by channel effects, phonetic variability, as a factor that causes session variability, is still a matter to be considered. We propose in this paper a new i-vector extraction algorithm from the total factor matrix which we term component reduction analysis (CRA). This new algorithm contributes to better modelling of session variability in the total factor space. We reported results on the male English trials of the core condition of the NIST 2008 Speaker Recognition Evaluation (SREs) dataset. As measured both by equal error rate and the minimum values of the NIST detection cost function, 10–15 % relative improvement is achieved compared to the baseline of traditional i-vector-based system.

The Gaussian mixture models (GMM) represent an efficient model that was broadly used in most of speaker recognition applications. This study introduces a novel method for speaker verification task. We propose a reduced feature vector employing new information detected from the speaker's voice for performing text-independent speaker verification applications using GMM. We use the power spectrum density of the speech signal to improve the system's performance. Speaker verification experiments were evaluated with the TIMIT dataset. The suggested system performance is evaluated against the baseline systems. The decrease in the error rate is well observed and the results have demonstrated the effectiveness of the new approach which avoids the use of more complex algorithms or the combination of different approaches.