Modulation features for noise robust speaker identification

Speech activity detection (SAD) on channel transmissions is a critical preprocessing task for speech, speaker and language recognition or for further human analysis. This paper presents a feature combination approach to improve SAD on highly channel degraded speech as part of the Defense Advanced Research Projects Agency’s (DARPA) Robust Automatic Transcription of Speech (RATS) program. The key contribution is the feature combination exploration of different novel SAD features based on pitch and spectro-temporal processing and the standard Mel Frequency Cepstral Coefficients (MFCC) acoustic feature. The SAD features are: (1) a GABOR feature representation, followed by a multilayer perceptron (MLP); (2) a feature that combines multiple voicing features and spectral flux measures (Combo); (3) a feature based on subband autocorrelation (SAcC) and MLP postprocessing and (4) a multiband comb-filter F0 (MBCombF0) voicing measure. We present single, pairwise and all feature combinations, show high error reductions from pairwise feature level combination over the MFCC baseline and show that the best performance is achieved by the combination of all features. Index Terms: speech detection, channel-degraded speech, robust voicing features

In this paper we propose softSAD: the direct integration of speech posteriors into a speaker recognition system as an alternative to using speech activity detection (SAD). Motivated by the need to use audio from short recordings more efficiently, softSAD removes the need to discard audio using speech/non-speech decisions based on a threshold as done with SAD. Instead, softSAD explicitly integrates into the Baum-Welch statistics a speech posterior for each frame. We compare softSAD and SAD in mismatched conditions by evaluating a system developed for the National Institute for Standards and Technology (NIST) 2012 speaker recognition evaluation (SRE) on the short test conditions of the channel-degraded Robust Automatic Transcription of Speech (RATS) speaker identification task (and vice versa). We demonstrate that softSAD provides benefit over SAD for short test audio in mismatched conditions.

This paper presents a robust speaker identification method from degraded speech signals. This proposed method depends on the Mel-frequency cepstral coefficients (MFCCs) for feature extraction from the degraded speech and its discrete cosine transform (DCT). It is known that the MFCCs based speech recognition methods are not robust enough in the presence of noise and channel degradation. So, the feature extraction from the DCT of the signal will assist in achieving a higher recognition rate. The artificial neural network (ANN) classification technique is used in the proposed method. The comparison between the proposed method and the method using the MFCCs only for feature extraction from noisy speech signals and telephone-like degraded signals shows that the proposed method improves the recognition rate in the presence of noise or degradation.

The i-vector representation has become increasingly popular in speaker and language recognition systems. The estimation of the projection matrix of the i-vector model is usually performed using the iterative expectation maximization (EM) algorithm. This work presents a novel approach to estimate the projection matrix of the i-vector representation and to estimate the i-vector representation for each utterance. In this approach, we formulate the estimation of the projection matrix as a principal component analysis (PCA) problem. Using the relation between PCA and a linear Gaussian model trained using the EM algorithm, we show that an approximate solution of the i-vector estimation can be obtained as the solution of a PCA problem. We evaluate the performance of our approximate i-vector estimation on the language recognition task of the robust automatic transcription of speech (RATS) project. The proposed approach reduces by 50% relative the computational time required to estimate the i-vector projection matrix and by 42% relative the computational time to estimate the i-vector representation compared to the standard EM-based approach to i-vector estimation. In addition, our experiments show improvements up to 29% relative in language recognition performance in terms of equal error rate compared to the standard EM-based i-vector estimation.

The acoustic classification technology of moving vehicles is a significant application in wireless sensor networks (WSNs). However, the wild environment makes it more intractable for the acoustic target classification owing to the complicated interference noise. The widely used mel-frequency cepstral coefficients (MFCCs) are somewhile contaminated by the acoustic noise, resulting in degrading the classification performance in real environments. To increase the classification accuracy of the moving vehicles, this paper presents an acoustic feature extraction method, which integrates the nonlinear function with the MFCC method, termed as NMFCC. Through nonlinear transformation, the proposed method could amplify the acoustic features in a nonlinear way, and obtain more robust acoustic features, thereby achieving a superior classification performance. Comparing the NMFCC with the baseline feature extraction method, experimental results not only demonstrate the viability of the proposed algorithm but also show a satisfactory performance of the moving vehicle classification for diverse practical scenarios in the wild.

This study investigates perceptual voice identification by analyzing the quantitative and qualitative acoustical characteristics listeners use to identify voices in non-native speech. Research on voice identification typically focuses on the direct quantitative comparison of voices to find unique identifiers [e.g., Morrison (2009) and Jessen (2008)]. However, in this study an experiment was run in order to analyze the acoustic cues people perceptually adhere to in identifying voices. Listeners of various native language backgrounds were asked to identify the voices of Mandarin Chinese speakers in both Mandarin and Mandarin-accented English speech. Listeners’ accuracy and speaker selections in fourteen voice line-ups (Sullivan and Schlichting 1998) were recorded. Speakers’ productions for all stimuli were analyzed using twelve acoustic features. The results find that though listeners were highly accurate at the voice recognition task, their errors do follow systematic trends. Quantitative and qualitative acoustic measures such as pitch, pitch variation, formant trajectories, intensity, and nasality prove to be reliable in patterning memorable and forgettable voices. That is, this study finds that certain acoustic characteristics are more salient in everyday voice identification.

Cepstral features and their derivatives have been successfully used in speaker recognitions tasks over the past two decades. Recently new features are emerging that make use of information that is not contained in cepstral features or their derivative, namely the phase spectra and instantaneous frequency. There are numerous approaches to capture instantaneous amplitude and frequency variations in the signal, resulting in an amplitude and frequency modulation (AM-FM) feature set. While the addition of modulation features has been shown to improve speech recognition systems based on cepstral features, their use in speaker recognition is new and has not been fully explored. This paper outlines some of these emerging features and their extraction methods for use in speaker recognition tasks. The paper also talks about the recently proposed Empirical Mode Decomposition (EMD) as an alternative method of extracting instantaneous frequency.

Recent research shows that the i-vector framework for speaker recognition can significantly benefit from phonetic information. A common approach is to use a deep neural network (DNN) trained for automatic speech recognition to generate a universal background model (UBM). Studies in this area have been done in relatively clean conditions. However, strong background noise is known to severely reduce speaker recognition performance. This study investigates a phonetically-aware i-vector system in noisy conditions. We propose a front-end to tackle the noise problem by performing speech separation and examine its performance for both verification and identification tasks. The proposed separation system trains a DNN to estimate the ideal ratio mask of the noisy speech. The separated speech is then used to extract enhanced features for the i-vector framework. We compare the proposed system against a multi-condition trained baseline and a traditional GMM-UBM i-vector system. Our proposed system provides an absolute average improvement of 8% in identification accuracy and 1.2% in equal error rate.

In this paper, we propose VoiceID loss, a novel loss function for training a speech enhancement model to improve the robustness of speaker verification. In contrast to the commonly used loss functions for speech enhancement such as the L2 loss, the VoiceID loss is based on the feedback from a speaker verification model to generate a ratio mask. The generated ratio mask is multiplied pointwise with the original spectrogram to filter out unnecessary components for speaker verification. In the experiments, we observed that the enhancement network, after training with the VoiceID loss, is able to ignore a substantial amount of time-frequency bins, such as those dominated by noise, for verification. The resulting model consistently improves the speaker verification system on both clean and noisy conditions.

Speaker identification is a biometric technique of determining an unknown speaker's identity among a number of speakers using distinguish latent information of uttered speech. Crime investigation, security control, telephone banking and trading, and information reservation are some applications of this technique. Frequency Domain Linear Prediction (FDLP) is a time-frequency-based feature has been derived using 2-D autoregressive model. This feature was constructed from sub-bands short frame energies estimation. FDLP has been used in this study to propose a robust text-dependent speaker identification technique. The clean features were used to obtain speaker behavioural model. Support vector machine has been used to train the proposed method. This presented study was tested in both clean and noisy conditions to validate the method extensively. The proposed method got significant improved performance over all traditional methods performances in noisy conditions. The obtained performance was indicated; the proposed method was very robust to noises and showed consistent performance irrespective to noises.

A method is proposed for superresolving multichannel data with application to video sequences. Based on a generalization of Papoulis’ sampling theorem, nonuniform samples of multiple channels are merged to generate high-resolution data. To overcome sampling ill posedness in the presence of noise, image frames are projected from standard orthonormal bases onto optimal Riesz bases defined by channel point spread functions (PSF’s). The method is therefore designed to perform under practical conditions of noise and other degradations. Unlike existing methods, where empirical models such as Gaussian, sinc, etc., are commonly used for characterizing channel PSF’s, the PSF’s are assumed unknown and possibly different and hence are blindly estimated from the observed data. The estimated PSF’s are then used to construct biorthogonal projection filters for the superresolution algorithm. This approach gives rise to a closed-form solution leading to a high-speed algorithm. The method has been tested and verified on PREDATOR video sequences (PREDATOR data are airborne video sequences of the Defense Advanced Research Projects Agency obtained by unmanned aircrafts).

Multi-task learning (MTL) and attention mechanism have been proven to effectively extract robust acoustic features for various speech-related tasks in noisy environments. In this study, we propose an attention-based MTL (ATM) approach that integrates MTL and the attention-weighting mechanism to simultaneously realize a multi-model learning structure that performs speech enhancement (SE) and speaker identification (SI). The proposed ATM system consists of three parts: SE, SI, and attention-Net (AttNet). The SE part is composed of a long-short-term memory (LSTM) model, and a deep neural network (DNN) model is used to develop the SI and AttNet parts. The overall ATM system first extracts the representative features and then enhances the speech signals in LSTM-SE and specifies speaker identity in DNN-SI. The AttNet computes weights based on DNN-SI to prepare better representative features for LSTM-SE. We tested the proposed ATM system on Taiwan Mandarin hearing in noise test sentences. The evaluation results confirmed that the proposed system can effectively enhance speech quality and intelligibility of a given noisy input. Moreover, the accuracy of the SI can also be notably improved by using the proposed ATM system.

Background/Objectives: Accurate nuclear segmentation is a fundamental step in computational pathology, enabling reliable estimation of cellularity and nuclear morphology. However, segmentation models are typically evaluated under ideal imaging conditions, while real-world microscopy data are affected by staining variability, noise, and image degradation. This study aims to comparatively evaluate three representative convolutional architectures for nuclei segmentation, with emphasis on robustness and clinical relevance under perturbed imaging conditions. Methods: U-Net, Attention U-Net, and U-Net++ were trained and evaluated on the BBBC038 nuclei microscopy dataset using fixed train–validation–test splits. Robustness was assessed under three types of synthetic perturbations: Gaussian blur, additive noise, and color jitter. Segmentation performance was quantified using the Dice coefficient and Intersection-over-Union (IoU). Paired Wilcoxon signed-rank tests with Holm correction and Cliff’s delta were used for statistical comparison. In addition, clinically relevant nuclear descriptors—nuclear count, median nuclear area, area interquartile range (IQR), and nuclear density—were extracted from predicted masks, and descriptor stability was analyzed as relative deviation from clean conditions. Results: Under clean imaging conditions, Attention U-Net achieved the highest mean Dice score, while paired statistical analysis indicated that U-Net++ exhibited the most consistent performance across test samples. Under image perturbations, Attention U-Net demonstrated greater robustness to blur and noise, whereas U-Net++ showed superior stability under color variations. Descriptor-based analysis further indicated that U-Net++ preserved nuclear count and density most reliably under chromatic perturbations, while U-Net exhibited larger instability in nuclear count and density, particularly under noise. Conclusions: Architectural design choices strongly influence not only pixel-level segmentation accuracy but also the stability of clinically relevant nuclear morphology descriptors. Robustness evaluation under multiple perturbation types reveals important trade-offs between architectures that are not captured by clean-image benchmarks alone. These findings highlight the necessity of multi-level evaluation strategies combining overlap metrics, statistical testing, robustness analysis, and descriptor stability assessment for future benchmarking and clinically reliable deployment of nuclei segmentation systems.

Residual networks for text-independent speaker identification: Unleashing the power of residual learning

In this paper, we study robust speaker recognition in far-field microphone situations. Two approaches are investigated to improve the robustness of speaker recognition in such scenarios. The first approach applies traditional techniques based on acoustic features. We introduce reverberation compensation as well as feature warping and gain significant improvements, even under mismatched training-testing conditions. In addition, we performed multiple channel combination experiments to make use of information from multiple distant microphones. Overall, we achieved up to 87.1% relative improvements on our Distant Microphone database and found that the gains hold across different data conditions and microphone settings. The second approach makes use of higher-level linguistic features. To capture speaker idiosyncrasies, we apply n-gram models trained on multilingual phone strings and show that higher-level features are more robust under mismatching conditions. Furthermore, we compared the performances between multilingual and multiengine systems, and examined the impact of a number of involved languages on recognition results. Our findings confirm the usefulness of language variety and indicate a language independent nature of this approach, which suggests that speaker recognition using multilingual phone strings could be successfully applied to any given language.