From Speaker Verification to Multispeaker Speech Synthesis, Deep Transfer with Feedback Constraint

Speaker verification (SV) aims to determine whether the speaker's identity of a test utterance is the same as the reference speech. In the past few years, extracting speaker embeddings using deep neural networks for SV systems has gone mainstream. Recently, different attention mechanisms and Transformer networks have been explored widely in SV fields. However, utilizing the original Transformer in SV directly may have frame-level information waste on output features, which could lead to restrictions on capacity and discrimination of speaker embeddings. Therefore, we propose an approach to derive utterance-level speaker embeddings via a Transformer architecture that uses a novel loss function named diffluence loss to integrate the feature information of different Transformer layers. Therein, the diffluence loss aims to aggregate frame-level features into an utterance-level representation, and it could be integrated into the Transformer expediently. Besides, we also introduce a learnable mel-fbank energy feature extractor named time-domain feature extractor that computes the mel-fbank features more precisely and efficiently than the standard mel-fbank extractor. Combining Diffluence loss and Time-domain feature extractor, we propose a novel Transformer-based time-domain SV model (DT-SV) with faster training speed and higher accuracy. Experiments indicate that our proposed model can achieve better performance in comparison with other models.

Privacy is the major concern that comes to the user's mind before sharing their data. There are various methods proposed in literature for providing privacy to speech data. Previous works that have been done to protect the speaker identity were done for speech applications like automatic speech recognition (ASR), speech analysis, etc. For these applications the presence of speaker identity is not essential while processing. The objective of this work is to provide privacy to the task in which presence of speaker identity is essential at the time of processing. In this work, privacy is provided to the speaker identity information present in speech signals while performing automatic speaker verification (ASV) tasks. In order to achieve the same, this work proposes a prosody modification based approach. The proposed approach is able to conceal the speaker identity from human perception by changing the pitch of the speech utterances with a pitch modification factor of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\alpha\geq 1$</tex> But at the same time the ASV system provides consistent performance irrespective of the change in pitch (i.e. for <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\alpha\geq 1)$</tex> . The same evidence has been shown through experiments in TIMIT and IITG-MV databases. A subjective study has also performed to verify the extent of speaker anonymization with respect to humans. The subjective study evaluates the performance in terms of mean opinion score (MOS). The observed MOS signifies the ability of the proposed approach to conceal the speaker's identity.

The speaker identity variability effect (SIVE) has long been interpreted as challenging the abstract phonetic representation hypothesis and supporting the episodic representation hypothesis. However, this review integrates evidence from the auditory system to argue that SIVE does not preclude the hypothesis of abstract representational coding. Speaker identity information is encoded in the primary auditory cortex, but changes in speaker identity can disrupt the continuity of auditory objects, thereby reducing phonetic processing efficiency. This effect is attributed to the dynamic allocation of attentional resources between speaker identification and phonetic decoding processes, supporting the hypothesis of independent pathways for speaker identity and speech processing. Future research should use functional magnetic resonance imaging technology to dissect how the dynamic coordination of neural networks handles speaker identity variability and to elucidate the roles of different attentional subsystems in adapting to such variability.

Speaker Identification systems have become area of intense research in recent years due to its wide range of applications in voice matching, biometric identification, mobile access security, health care management and transportation. The major challenge in speaker identification is how to provide robust and computationally efficient features that accurately captures speaker's unique identity for higher generalized identification. This paper presents hierarchical classification approach for speaker identification using robust time domain features. In hierarchical classification approach, the first level classifier identifies the gender voice (i.e. male or female voice). In addition, the second level classifier identifies the specific speaker voice. The experiments were conducted using the speech data collected from ten subjects including male and female to ensure the generalized model evaluations. Several time domain features were extracted from the collected dataset that were proven highly discriminative for gender identification and specific speaker identification. The proposed method helps to reduce computational time with increase speaker recognition rate. The experimental results obtained highest accuracy of 96.9% using random forest classifier for gender identification. Moreover, for specific speaker identification the highest accuracy of 78% was observed in male speaker identification and 88.7% accuracy was obtained in female speaker identification using random forest classifier. The encouraging results of our experiments show the impact of hierarchical speaker identification using carefully selected time domain features.

In this paper, we present a novel system for joint speaker identification and speech separation. For speaker identification a single-channel speaker identification algorithm is proposed which provides an estimate of signal-to-signal ratio (SSR) as a by-product. For speech separation, we propose a sinusoidal model-based algorithm. The speech separation algorithm consists of a double-talk/single-talk detector followed by a minimum mean square error estimator of sinusoidal parameters for finding optimal codevectors from pre-trained speaker codebooks. In evaluating the proposed system, we start from a situation where we have prior information of codebook indices, speaker identities and SSR-level, and then, by relaxing these assumptions one by one, we demonstrate the efficiency of the proposed fully blind system. In contrast to previous studies that mostly focus on automatic speech recognition (ASR) accuracy, here, we report the objective and subjective results as well. The results show that the proposed system performs as well as the best of the state-of-the-art in terms of perceived quality while its performance in terms of speaker identification and automatic speech recognition results are generally lower. It outperforms the state-of-the-art in terms of intelligibility showing that the ASR results are not conclusive. The proposed method achieves on average, 52.3% ASR accuracy, 41.2 points in MUSHRA and 85.9% in speech intelligibility.

The paper presents an unsupervised binaural scene analyzer that is capable of performing simultaneous operations, such as localization, detection and recognition of target speaker amidst in a reverberant and noise inferential environment. The proposed technique includes three main stages: sound source localization, speaker recognition and auditory saliency-based indexing system. The stage one involves the localization of target speaker by incorporating the binaural cues into Azimuth dependant GMM-EM classifier. During the second stage, the study proposes a Gabor-Hilbert Envelope Coefficient (GHEC) based spectro-temporal extractor as an efficient speaker recognition method which shows better robustness with minimum computational complexity. The Hilbert Envelope provides relevant acoustic information and also it improvises the performance of speaker identification process in different reverberant environments and SNR values. Later in the third stage, the auditory saliency based diarization is proposed as a process of indexing the speech contents based on the image identity of the target speaker. The proposed system may be used to catalogue the entire speech content with the corresponding image of the target speaker that finds wide range of applications, including teleconference, hands-free communication and meeting hall content localization and fast audio retrieval in a repository.

Multi-task learning (MTL) and attention mechanism have been proven to effectively extract robust acoustic features for various speech-related tasks in noisy environments. In this study, we propose an attention-based MTL (ATM) approach that integrates MTL and the attention-weighting mechanism to simultaneously realize a multi-model learning structure that performs speech enhancement (SE) and speaker identification (SI). The proposed ATM system consists of three parts: SE, SI, and attention-Net (AttNet). The SE part is composed of a long-short-term memory (LSTM) model, and a deep neural network (DNN) model is used to develop the SI and AttNet parts. The overall ATM system first extracts the representative features and then enhances the speech signals in LSTM-SE and specifies speaker identity in DNN-SI. The AttNet computes weights based on DNN-SI to prepare better representative features for LSTM-SE. We tested the proposed ATM system on Taiwan Mandarin hearing in noise test sentences. The evaluation results confirmed that the proposed system can effectively enhance speech quality and intelligibility of a given noisy input. Moreover, the accuracy of the SI can also be notably improved by using the proposed ATM system.

Speaker diarization is a task to identify the speaker when different speakers spoke in an audio or video recording environment. Artificial intelligence (AI) fields have effectively used Deep Learning (DL) to solve a variety of real-world application challenges. With effective applications in a wide range of subdomains, such as natural language processing, image processing, computer vision, speech and speaker recognition, and emotion recognition, cyber security, and many others, DL, a very innovative field of Machine Learning (ML), that is quickly emerging as the most potent machine learning technique. DL techniques have outperformed conventional approaches recently in speaker diarization as well as speaker recognition. The technique of assigning classes to speech recordings that correspond to the speaker's identity is known as speaker diarization, and it allows one to determine who spoke when. A crucial step in speech processing is speaker diarization, which divides an audio recording into different speaker areas. In-depth analysis of speaker diarization utilizing a variety of deep learning algorithms that are presented in this research paper. NIST-2000 CALLHOME and our in-house database ALSD-DB are the two voice corpora we used for this study's tests. TDNN-based embeddings with x-vectors, LSTM-based embeddings with d-vectors, and lastly embeddings fusion of both x-vector and d-vector are used in the tests for the basic system. For the NIST-2000 CALLHOME database, LSTM based embeddings with d-vector and embeddings integrating both x-vector and d-vector exhibit improved performance with DER of 8.25% and 7.65%, respectively, and of 10.45% and 9.65% for the local ALSD-DB database

Automatic speaker recognition using time alignment of spectrograms

Speaker Identity and Voice Quality: Modeling Human Responses and Automatic Speaker Recognition

Human-computer interaction for virtual-real fusion

Various methods have been proposed for protecting the speaker's identity while preserving speech intelligibility. However, existing studies fail to consider the overall tradeoff between speech utility, speaker verification, and inference of voice physical attributes, such as emotion, age, accent, and gender. we propose a tradeoff metric to encapsulate voice biometrics as well as different voice attributes, to study the feasibility of applying cutting-edge voice anonymization solutions to achieve the optimum tradeoff between privacy protection and speech utility.

This article explores a method for speaker verification to appropriately set the threshold value used to judge the identities of individual speakers and also considers methods to update speaker models and the robustness of the speaker models so as to make the models more robust with respect to utterance variations, using a small amount of data for updating that was recently uttered. The speaker model is represented by the hidden Markov model (HMM). In the model updating, the parameters of the speaker HMM are estimated from the data for updating and the current parameter values. For setting the threshold, the new threshold value for each speaker is set with the initial value as a value that is passed to an FA rate higher than the equal error rate (a value for which the false rejection rate and the false acceptance rate are equal), which is calculated from the data for updating and which steadily approaches the value that was passed to the equal error rate in concert with updating of the speaker HMM. The results of evaluating this method using text-dependent and text-prompted speaker verification experiments with twenty speakers shows that the average error rate fell by roughly 40% for the text-independent type and by roughly 80% for the text-prompted type as compared to when the model and threshold value are not updated. © 1999 Scripta Technica, Syst Comp Jpn, 30(13): 96–105, 1999

Voice cloning has become a rapidly evolving field in artificial intelligence and speech processing. Recent advances in deep learning have made it possible to replicate human voices with remarkable accuracy using relatively small datasets. At the core of this technology lies the ability to analyze and interpret audio signals in order to capture the unique characteristics of a speaker’s voice. Audio files contain a wide range of information including speech content, speaker identity, emotional state, pronunciation patterns, and environmental context. Extracting and modeling this information is essential for developing effective voice cloning systems. Modern voice synthesis frameworks rely on several stages of audio signal processing including signal acquisition, preprocessing, feature extraction, representation learning, and neural speech generation. This paper presents a comprehensive study of how audio signals are processed in voice cloning systems and explores the various types of information that can be extracted from audio recordings. The research examines the structure of digital audio signals, the methods used to convert sound waves into machine-readable data, and the feature extraction techniques that capture acoustic properties of speech. In addition, the paper investigates modern neural architectures used for voice cloning such as spectrogram-based models, neural vocoders, and speaker embedding networks. The study also highlights several practical applications of voice cloning technologies including digital assistants, personalized speech synthesis, accessibility tools, and entertainment systems. Furthermore, the research discusses ethical considerations and potential risks associated with voice cloning technologies, emphasizing the need for responsible development and robust detection mechanisms. The findings demonstrate that audio signals contain rich multi-layered information that can be effectively utilized to develop advanced speech synthesis and analysis systems

Speaker identification is a biometric technique of determining an unknown speaker's identity among a number of speakers using distinguish latent information of uttered speech. Crime investigation, security control, telephone banking and trading, and information reservation are some applications of this technique. Frequency Domain Linear Prediction (FDLP) is a time-frequency-based feature has been derived using 2-D autoregressive model. This feature was constructed from sub-bands short frame energies estimation. FDLP has been used in this study to propose a robust text-dependent speaker identification technique. The clean features were used to obtain speaker behavioural model. Support vector machine has been used to train the proposed method. This presented study was tested in both clean and noisy conditions to validate the method extensively. The proposed method got significant improved performance over all traditional methods performances in noisy conditions. The obtained performance was indicated; the proposed method was very robust to noises and showed consistent performance irrespective to noises.