Optimization of the parameters characterizing sigmoidal rate-level functions based on acoustic features

It is widely known that the quality of confidence measure is critical for speech applications. In this paper, we present our recent work on improving word confidence scores by calibrating them using a small set of calibration data when only the recognized word sequence and associated raw confidence scores are made available. The core of our technique is the maximum entropy model with distribution constraints which naturally and effectively make use of the word distribution, the raw confidence-score distribution, and the context information. We demonstrate the effectiveness of our approach by showing that it can achieve relative 38% mean square error (MSE), 39% negative normalized likelihood (NNLL), and 23% equal error rate (EER) reduction on a voice mail transcription data set and relative 35% MSE, 45% NNLL, and 35% EER reduction on a command and control data set.

Most of the speaker recognition systems use system features for speaker recognition which are mostly spectral in nature. Recently, there has been significant work on using source features, viz., prosodies and pitch dynamics, glottal flow derivative, Linear Prediction (LP) residual and its phase, wavelet-domain representation of LP residual, etc for speaker recognition. In this paper, a new source-like feature set, viz., Teager Energy Operator (TEO) Phase is developed for speaker recognition. Proposed TEO Phase has several salient features as compared to that of LP residual phase, viz., less number of spurious peaks in Hilbert envelope plot, and better perceptual and sample correlation with speech signal. Furthermore, this avoids use of voiced/unvoiced detection, preprocessing techniques such as windowing and pre-emphasis and LP residual computation. Experiments have been carried out for speaker recognition task using TEO Phase and LP residual phase with polynomial classifier of 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nd</sup> order approximation. For speaker verification, a reduction in equal error rate (EER) by 2.62 % over LP residual phase is achieved for proposed feature set.

We discuss the limitations of the i-vector representation of speech segments in speaker recognition and explain how Joint Factor Analysis (JFA) can serve as an alternative feature extractor in a variety of ways. Building on the work of Zhao and Dong, we implemented a variational Bayes treatment of JFA which accommodates adaptation of universal background models (UBMs) in a natural way. This allows us to experiment with several types of features for speaker recognition: speaker factors and diagonal factors in addition to i-vectors, extracted with and without UBM adaptation in each case. We found that, in text-independent speaker verification experiments on NIST data, extracting i-vectors with UBM adaptation led to a 10% reduction in equal error rates although performance did not improve consistently over the whole DET curve. We achieved a further 10% reduction (with a similar inconsistency) by using speaker factors extracted with UBM adaptation as features. In text-dependent speaker recognition experiments on RSR2015 data, we were able to achieve very good performance using a JFA model with diagonal factors but no speaker factors as a feature extractor. Contrary to standard practice, this JFA model was configured so as to model speakerphrase combinations (rather than speakers) and it was trained on utterances of very short duration (rather than whole recording sessions). We also present a variant of the length normalization trick inspired by uncertainty propagation which leads to substantial gains in performance over the whole DET curve.

This study proposes an i-Vector based approach to gender identification. Gender-labeled utterances from the Fisher English (FE) corpus are used to formulate an i-Vector extraction framework, and a Probabilistic Linear Discriminant Analysis (PLDA) back-end is employed to compute the scores for gender identification. A novel duration mismatch compensation strategy is also presented that offers very little degradation in identification accuracy even with a large reduction in the duration of the test-segment. The proposed method is shown to consistently outperform a GMM-UBM based gender-identification scheme on several test-sets created from a held-out portion of the FE corpus, and is able to achieve an identification accuracy of up to 97.63%. On the severely distorted and multilingual DARPA-RATS (Robust Automatic Transcription of Speech) corpora, the proposed approach achieves an identification accuracy of 76.48% using only the FE data in training. Next, a novel unsupervised domain adaptation strategy is also presented that utilizes only unlabeled RATS data to adapt the out-of-domain PLDA parameters derived from the FE training data. The strategy is able to offer a 6.8% relative improvement in identification accuracy, and a 14.75% relative reduction in Equal Error Rate (EER) compared to using the out-of-domain PLDA model on the RATS test-utterances. These improvements are significant since: 1) RATS test-utterances are severely distorted, 2) No labeled data of any kind is used for 4 of the 5 languages present in the test-utterances.

In this paper appropriate confidence measures (CMs) are investigated for Mandarin command word recognition, both in the so-called target region and non-target region, respectively. Here the target region refers to the recognized speech part of command word while the non-target region refers to the recognized silence part. It shows that exploiting extra information in the non-target region can effectively complement the traditional CM which usually focus on the target region. Furthermore, when analyzing the non-target region in a more theoretical way, where Bayesian information criterion (BIC) is employed to locate more precise boundary in the non-target region, even more improvement is achieved. In two different Mandarin telephone command word tasks, more than 20% relative reduction of equal error rate (EER) is obtained.

In this paper, we present a Mixture Linear Prediction based approach for robust Gammatone Cepstral Coefficients extraction (MLPGCCs). The proposed method provides performance improvement of Automatic Speaker Verification (ASV) using i-vector and Gaussian Probabilistic Linear Discriminant Analysis GPLDA modeling under transmission channel noise. The performance of the extracted MLPGCCs was evaluated using the NIST 2008 database where a single channel microphone recorded conversational speech. The system is analyzed in the presence of different channel transmission noises such as Additive White Gaussian (AWGN) and Rayleigh fading at various Signals to Noise Ratio (SNR) levels. The evaluation results show that the MLPGCCs features are a promising way for the ASV task. Indeed, the speaker verification performance using the MLPGCCs proposed features is significantly improved compared to the conventional Gammatone Frequency Cepstral Coefficients (GFCCs) and Mel Frequency Cepstral Coefficients (MFCCs) features. For speech signals corrupted with AWGN noise at SNRs ranging from (-5 dB to 15 dB), we obtain a significant reduction of the Equal Error Rate (EER) ranging from 9.41% to 6.65% and 3.72% to 1.50%, compared with conventional MFCCs and GFCCs features respectively. In addition, when the test speech signals are corrupted with Rayleigh fading channel we achieve an EER reduction ranging from 23.63% to 7.8% and from 10.88% to 6.8% compared with conventional MFCCs and GFCCs, respectively. We also found that the combination of GFCCs and MLPGCCs gives the highest performance of speaker verification system. The best performance combination achieved is around EER from 0.43% to 0.59% and 1.92% to 3.88%.

In this paper, we propose a dynamic in-search data selection method to diagnose competing information automatically from speech data. In our method, the Viterbi beam search is used to decode all training data. During decoding, all partial paths within the beam are examined to identify the so-called competing-token and true-token sets for each individual hidden Markov model (HMM). In this work, the collected data tokens are used for acoustic modeling and utterance verification as two specific examples. In acoustic modeling, the true-token sets are used to adapt HMMs with a sequential maximum a posteriori adaptation method, while a generalized probabilistic descent-based discriminative training method is proposed to improve HMMs based on competing-token sets. In utterance verification, under the framework of likelihood ratio testing, the true-token sets are employed to train positive models for the null hypothesis and the competing-token sets are used to estimate negative models for the alternative hypothesis. All the proposed methods are evaluated in Bell Laboratories communicator system. Experimental results show that the new acoustic modeling method can consistently improve recognition performance over our best maximum likelihood estimation models, roughly 1% absolute reduction in word error rate. The results also show the new verification models can significantly improve the performance of utterance verification over the conventional anti models, almost relatively 30% reduction of equal error rate when identifying misrecognized words from the recognition results.

The automatic speaker verification (ASV) has achieved significant progress in recent years. However, it is still very challenging to generalize the ASV technologies to new, unknown and spoofing conditions. Most previous studies focused on extracting the speaker information from natural speech. This paper attempts to address the speaker verification from another perspective. The speaker identity information was exploited from singing speech. We first designed and released a new corpus for speaker verification based on singing and normal reading speech. Then, the speaker discrimination was compared and analyzed between natural and singing speech in different feature spaces. Furthermore, the conventional Gaussian mixture model, the dynamic time warping and the state-of-the-art deep neural network were investigated. They were used to build text-dependent ASV systems with different training-test conditions. Experimental results show that the voiceprint information in the singing speech was more distinguishable than the one in the normal speech. More than relative 20% reduction of equal error rate was obtained on both the gender-dependent and independent 1 s-1 s evaluation tasks.

Speaker verification technology has widespread applications across many domains, benefiting from deep learning advancements. However, due to the high cost of acquiring labeled data, semi-supervised learning has emerged as a prominent research focus. Current semi-supervised learning frameworks commonly suffer from two limitations: (1) the labeled data distribution is often restricted, and (2) they still rely on a considerable amount of labeled data. To address these issues, we propose three different distribution scenarios of labeled data and construct a general semi-supervised framework. Furthermore, to enhance the guidance efficacy of limited labeled data, we innovatively employ prototype space optimization to strengthen the model's discriminative capability under low-resource scenarios. Experimental results demonstrate that on the Vox1-o test set, our approach achieves a 41.7% relative reduction in equal error rate compared to self-supervised baselines, and a 28.9% improvement over conventional semi-supervised framework baselines.

For the first time in this paper we present results showing the effect of out of plane speaker head pose variation on a lip biometric based speaker verification system. Using appearance DCT based features, we adopt a Mutual Information analysis technique to highlight the class discriminant DCT components most robust to changes in out of plane pose. Experiments are conducted using the initial phase of a new multi view AudioVisual database designed for research and development of poseinvariant speech and speaker recognition. We show that verification performance can be improved by substituting higher order horizontal DCT components for vertical, particularly in the case of a train/test pose angle mismatch. We show that the best performance can be achieved by combining this alternative feature selection with multi view training, reporting a relative 45% Equal Error Rate reduction over a common energy based selection.

We propose a new approach to the problem of estimating the hyperparameters which define the interspeaker variability model in joint factor analysis. We tested the proposed estimation technique on the NIST 2006 speaker recognition evaluation data and obtained 10%-15% reductions in error rates on the core condition and the extended data condition (as measured both by equal error rates and the NIST detection cost function). We show that when a large joint factor analysis model is trained in this way and tested on the core condition, the extended data condition and the cross-channel condition, it is capable of performing at least as well as fusions of multiple systems of other types. (The comparisons are based on the best results on these tasks that have been reported in the literature.) In the case of the cross-channel condition, a factor analysis model with 300 speaker factors and 200 channel factors can achieve equal error rates of less than 3.0%. This is a substantial improvement over the best results that have previously been reported on this task.

Intelligibility enhancement of HMM-generated speech in additive noise by modifying Mel cepstral coefficients to increase the glimpse proportion

Digital voice analysis is an emerging tool for differentiating cognitive states, but it poses privacy risks as automated systems may inadvertently identify speakers. We developed a computational framework to evaluate the trade-off between voice obfuscation and cognitive assessment accuracy, using pitch-shifting as a representative method. This framework was applied to voice recordings from the Framingham Heart Study (FHS, n=128) and the DementiaBank Delaware (DBD, n=85) corpus, both featuring responses to neuropsychological tests. Speaker obfuscation was measured via equal error rate (EER), and diagnostic utility was assessed through machine learning models distinguishing cognitive states: normal cognition (NC), mild cognitive impairment (MCI), and dementia (DE). With the top 20 acoustic features, our framework achieved classification accuracies of 62.2% (EER: 0.3335) on the FHS dataset for NC, MCI, and DE differentiation, and 63.7% (EER: 0.1796) on the DBD dataset for NC and MCI differentiation, using obfuscated speech files. Our results demonstrate the feasibility of privacy-preserving voice markers, offering a scalable solution for voice-based cognitive assessments. We developed a computational framework using pitch-shifting and acoustic transformations to balance speaker privacy and diagnostic utility in voice-based cognitive assessments. We evaluated the framework on two independent datasets, Framingham Heart Study (FHS, n=128) and DementiaBank Delaware (DBD, n=85) corpus, assessing the trade-off between privacy (measured by equal error rate [EER]) and classification accuracy. Our framework achieved classification accuracies of 62.2% (EER: 0.3335) for distinguishing normal cognition (NC), mild cognitive impairment (MCI), and dementia in the FHS dataset and 63.7% (EER: 0.1796) for NC and MCI differentiation in the DBD dataset, using obfuscated speech files. Our framework demonstrates that pitch-shifting levels can preserve diagnostic utility while protecting speaker identity, offering a scalable and privacy-preserving solution.

Advances in synthetic speech generation pose significant risks to voice communication security. This study proposes an empirical framework for synthetic speech detection with an emphasis on cross-language robustness. Acoustic and prosodic features, including MFCC, CQCC, pitch, jitter, and shimmer, are evaluated using classical machine learning methods and deep learning models. Experiments are conducted on the FoR (monolingual) and ODSS (multilingual) benchmark datasets using stratified 80%–20% train–test splits and stratified 5-fold cross-validation. On FoR, a CNN achieves 99.2% accuracy with a 0.75% Equal Error Rate (EER). On ODSS, ResNet reaches 96.5% accuracy and 3.72% EER under the percentage split. Additionally, Random Forest attains 100% accuracy and 0% EER in 5-fold cross-validation on ODSS, indicating strong feature separability while suggesting potential overfitting effects. The results demonstrate that the proposed hybrid features and deep architectures provide high generalizability across different languages and synthesis techniques.

Shells of planktonic foraminifera recovered from marine sediments provide a multitude of important palaeoproxies. Most of these proxies are based on the assumption that each morphospecies of planktonic foraminifera represents a genetically continuous species with a unique habitat. Recent discovery of hitherto hidden genetic diversity among modern planktonic foraminifera has significant repercussions on palaeoproxies derived from their fossil shells. We have compiled all available data on this genetic diversity. To date, 33 cryptic genetic types were found in 9 out of the 22 sequenced morphospecies of modern planktonic foraminifera. An examination of this database suggests that cryptic genetic diversity may be a prevalent pattern among modern planktonic foraminifera, but that the total number of cryptic genetic types per morphospecies is not large and that most genetic types show a non-random pattern of distribution in the oceans. Using modern distribution data from the Atlantic Ocean as constraints, the relationship between abundances of three genetic types of Globigerina bulloides and sea-surface temperature has been modelled and this model has been applied to a database of species counts in Atlantic coretops (761 samples). Trials with artificial neural networks (ANNs), the modern analogue technique and Imbrie-Kipp transfer functions showed that the splitting of G. bulloides into three genetic types resulted in substantial reduction in the prediction error rate (by 5 to 34%) and that this improvement was by far greatest in ANN trials (on average more than 20%). We conclude that such a large reduction in error rate occurred because the models resonated with a real pattern in the original data. This study indicates that genetic diversity among planktonic foraminifera may become more of a gift than malaise to palaeoproxies. If it becomes possible to distinguish these genetic types in the fossil record, the accuracy of proxies based on planktonic foraminifera will indeed substantially increase.