Arabic speaker recognition system based on phoneme fusion

Speaker and language recognition in noisy and degraded channel conditions continue to be a challenging problem mainly due to the mismatch between clean training and noisy test conditions. In the presence of noise, the most reliable portions of the signal are the high energy regions which can be used for robust feature extraction. In this paper, we propose a front end processing scheme based on autoregressive (AR) models that represent the high energy regions with good accuracy followed by a modulation filtering process. The AR model of the spectrogram is derived using two separable time and frequency AR transforms. The first AR model (temporal AR model) of the sub-band Hilbert envelopes is derived using frequency domain linear prediction (FDLP). This is followed by a spectral AR model applied on the FDLP envelopes. The output 2-D AR model represents a low-pass modulation filtered spectrogram of the speech signal. The band-pass modulation filtered spectrograms can further be derived by dividing two AR models with different model orders (cut-off frequencies). The modulation filtered spectrograms are converted to cepstral coefficients and are used for a speaker recognition task in noisy and reverberant conditions. Various speaker recognition experiments are performed with clean and noisy versions of the NIST-2010 speaker recognition evaluation (SRE) database using the state-of-the-art speaker recognition system. In these experiments, the proposed front-end analysis provides substantial improvements (relative improvements of up to 25%) compared to baseline techniques. Furthermore, we also illustrate the generalizability of the proposed methods using language identification (LID) experiments on highly degraded high-frequency (HF) radio channels and speech recognition experiments on noisy data.

Speech signals convey various yet mixed information ranging from linguistic to speaker-specific information. However, most of acoustic representations characterize all different kinds of information as whole, which could hinder either a speech or a speaker recognition (SR) system from producing a better performance. In this paper, we propose a novel deep neural architecture (DNA) especially for learning speaker-specific characteristics from mel-frequency cepstral coefficients, an acoustic representation commonly used in both speech recognition and SR, which results in a speaker-specific overcomplete representation. In order to learn intrinsic speaker-specific characteristics, we come up with an objective function consisting of contrastive losses in terms of speaker similarity/dissimilarity and data reconstruction losses used as regularization to normalize the interference of non-speaker-related information. Moreover, we employ a hybrid learning strategy for learning parameters of the deep neural networks: i.e., local yet greedy layerwise unsupervised pretraining for initialization and global supervised learning for the ultimate discriminative goal. With four Linguistic Data Consortium (LDC) benchmarks and two non-English corpora, we demonstrate that our overcomplete representation is robust in characterizing various speakers, no matter whether their utterances have been used in training our DNA, and highly insensitive to text and languages spoken. Extensive comparative studies suggest that our approach yields favorite results in speaker verification and segmentation. Finally, we discuss several issues concerning our proposed approach.

Although the field of automatic speaker or speech recognition has been extensively studied over the past decades, the lack of robustness has remained a major challenge. The missing data technique (MDT) is a promising approach. However, its performance depends on the correlation across frequency bands. This paper presents a new reconstruction method for feature enhancement based on the trait. In this paper, the degree of concentration across frequency bands is measured with principal component analysis (PCA). Through theoretical analysis and experimental results, it is found that the correlation of the feature vector extracted from the sub-band (SB) is much stronger than the ones extracted from the full-band (FB). Thus, rather than dealing with the spectral features as a whole, this paper splits full-band into sub-bands and then individually reconstructs spectral features extracted from each SB based on MDT. At the end, those constructed features from all sub-bands will be recombined to yield the conventional mel-frequency cepstral coefficient (MFCC) for recognition experiments. The 2-sub-band reconstruction approach is evaluated in speaker recognition system. The results show that the proposed approach outperforms full-band reconstruction in terms of recognition performance in all noise conditions. Finally, we particularly discuss the optimal selection of frequency division ways for the recognition task. When FB is divided into much more sub-bands, some of the correlations across frequency channels are lost. Consequently, efficient division ways need to be investigated to perform further recognition performance.

Speaker Recognition is the ability of the system to recognize the speaker from the set of speaker samples available in the system. It is of 2 types, one uses a keyword, called text-dependent systems, and another one can recognize the voice in any language/text, also called as text-independent speaker recognition. In this paper, a text-independent, language-independent speaker recognition system is implemented using dense & convolutional neural networks. Speaker recognition has found several applications in upcoming electronic products like personal/home assistants, telephone banking and biometric identification. In this paper, we explore a system that uses MFCC along with DNN and CNN as the model for building a speaker recognition system.

Speaker identification refers to the process of recognizing human voice using artificial intelligence techniques. Speaker identification technologies are widely applied in voice authentication, security and surveillance, electronic voice eavesdropping, and identity verification. In the speaker identification process, extracting discriminative and salient features from speaker utterances is an important task to accurately identify speakers. Various features for speaker identification have been recently proposed by researchers. Most studies on speaker identification have utilized short-time features, such as perceptual linear predictive (PLP) coefficients and Mel frequency cepstral coefficients (MFCC), due to their capability to capture the repetitive nature and efficiency of signals. Various studies have shown the effectiveness of MFCC features in correctly identifying speakers. However, the performances of these features degrade on complex speech datasets, and therefore, these features fail to accurately identify speaker characteristics. To address this problem, this study proposes a novel fusion of MFCC and time-based features (MFCCT), which combines the effectiveness of MFCC and time-domain features to improve the accuracy of text-independent speaker identification (SI) systems. The extracted MFCCT features were fed as input to a deep neural network (DNN) to construct the speaker identification model. Results showed that the proposed MFCCT features coupled with DNN outperformed existing baseline MFCC and time-domain features on the LibriSpeech dataset. In addition, DNN obtained better classification results compared with five machine learning algorithms that were recently utilized in speaker recognition. Moreover, this study evaluated the effectiveness of one-level and two-level classification methods for speaker identification. The experimental results showed that two-level classification presented better results than one-level classification. The proposed features and classification model for identifying a speaker can be widely applied to different types of speaker datasets.

ABSTRACT: Speech is one of the most natural and efficient forms of communication among humans, and it is becoming increasingly integral to human-computer interaction in modern applications such as virtual assistants, smart devices, and automated customer service systems. Among the various technologies that leverage speech, speaker recognition stands out as a biometric method that identifies or verifies individuals based on the unique characteristics of their voice. This approach has gained significant traction due to its non-intrusive nature, ease of integration, and applicability across various domains such as security, forensics, and personalised user experiences. In this paper, we introduce a novel speaker recognition system specifically designed to identify speakers based on utterances in the Marathi language, a linguistically rich and widely spoken regional language in India. The system utilises Mel-Frequency Cepstral Coefficients (MFCCS) to extract distinguishing vocal features that closely mimic the human auditory perception. MFCCS are particularly effective in capturing speech’s phonetic and acoustic properties, making them a preferred choice in speech and speaker recognition tasks. Vector quantisation (VQ) is applied to optimise the feature set and reduce computational complexity. VQ compresses the high-dimensional MFCC feature vectors into representative clusters without significantly compromising accuracy, thereby enhancing system efficiency. These features are then processed using hidden Markov Models (HMMS), which are adept at modelling temporal sequences and dynamic variations in speech. HMMS offer a statistical framework that effectively captures the sequential nature of speech patterns, leading to more reliable speaker modelling and recognition.

The paper presents an unsupervised binaural scene analyzer that is capable of performing simultaneous operations, such as localization, detection and recognition of target speaker amidst in a reverberant and noise inferential environment. The proposed technique includes three main stages: sound source localization, speaker recognition and auditory saliency-based indexing system. The stage one involves the localization of target speaker by incorporating the binaural cues into Azimuth dependant GMM-EM classifier. During the second stage, the study proposes a Gabor-Hilbert Envelope Coefficient (GHEC) based spectro-temporal extractor as an efficient speaker recognition method which shows better robustness with minimum computational complexity. The Hilbert Envelope provides relevant acoustic information and also it improvises the performance of speaker identification process in different reverberant environments and SNR values. Later in the third stage, the auditory saliency based diarization is proposed as a process of indexing the speech contents based on the image identity of the target speaker. The proposed system may be used to catalogue the entire speech content with the corresponding image of the target speaker that finds wide range of applications, including teleconference, hands-free communication and meeting hall content localization and fast audio retrieval in a repository.

This paper presents an experimental evaluation of the combined temporal and spectral processing methods for speaker recognition task under noise, reverberation or multi-speaker environments. Automatic speaker recognition system gives good performance in controlled environments. Speech recorded in real environments by distant microphones is degraded by factors like background noise, reverberation and interfering speakers. This degradation strongly affects the performance of the speaker recognition system. Combined temporal and spectral processing (TSP) methods proposed in our earlier study are used for pre-processing to improve the speaker-specific features and hence the speaker recognition performance. Different types of degradation like background noise, reverberation and interfering speaker are considered for evaluation. The evaluation is carried out for the individual temporal processing, spectral processing and the combined TSP method. The experimental results show that the combined TSP methods give relatively higher recognition performance compared to either temporal or spectral processing alone.

Speaker recognition is used today in a wide range of applications. The presence of reverberation, in hands-free systems for example, results in performance degradation. The effect of reverberation on the feature vectors and its relation to optimal GMM order are investigated. Optimal model order is calculated in terms of minimum BIC and KIC, and tested for EER of a GMM-based speaker recognition system. Experimental results show that for high reverberation time, reducing model order reduces EER values of speaker recognition. The effect of CMS on state of the art GMM and AGMM-based speaker recognition systems is investigated for reverberant speech. Results show that high reverberation time reduces the effectiveness of CMS.

In this paper, we study a front-end frame selection approach for the interview channel speaker recognition system. This new approach keeps the high quality speech frames and removes noisy and irrelevant speech frames for speaker modeling. For robust voice activity detection (VAD) under the different types of microphones located in the interview room, we adopt the spectral subtraction algorithm for noise reduction. An energy based frame selection algorithm is first applied to indicate the speech activity at the frame level. To overcome the summed channel effects in the interview condition, a study is conducted to effectively extract the relevant speaker's speech frames based on VAD Tags and ASR transcript Tags provided by NIST. The eigenchannel based GMM-SVM speaker recognition system is used to evaluate the proposed method. The experiments are conducted on the NIST 2008 and NIST 2010 Speaker Recognition Evaluation interview-interview conditions. It demonstrates that the approach provides an efficient way to select high quality speech frames and the relevant speaker's voice in the interview environment for speaker recognition.

Speaker recognition is an advanced method to identify a person from the biometric characteristics of speaking voice samples. Speaker recognition has become a vastly popular and useful research subject with countless essential applications in security, assistance, replication, authentication, automation, and verification. Many techniques are implemented using deep learning and neural network concepts and various datasets for speaker verification and identification. The primary goal of this work is to create improved robust techniques of speaker recognition to identify audio and enhance accuracy to human levels of comprehension. TIMIT and LibriSpeech datasets are used in this paper to develop an efficient automatic speaker recognition system. This work focuses on using MFCC to transform audio to spectrograms without losing the essential features of the audio file in question. We have used a closed set and an open set implementation procedure on these datasets. The closed set implementation uses a standard machine learning convention of utilizing the same datasets for training and testing, leading to higher accuracy. On the other hand, the open set implementation uses one dataset to train and another to test on each occasion. The accuracy, in this case, turned out to be relatively lower. On each dataset, CNN and LSTM deep learning techniques have been used to identify the sound, leading to the observation that implementing CNN resulted in a more significant accuracy.

Deep neural network based speaker recognition systems can easily be deceived by an adversary using minuscule imperceptible perturbations to the input speech samples. These adversarial attacks pose serious security threats to the speaker recognition systems that use speech biometric. To address this concern, in this work, we propose a new defense mechanism based on a hybrid adversarial training (HAT) setup. In contrast to existing works on countermeasures against adversarial attacks in deep speaker recognition that only use class-boundary information by supervised cross-entropy (CE) loss, we propose to exploit additional information from supervised and unsupervised cues to craft diverse and stronger perturbations for adversarial training. Specifically, we employ multi-task objectives using CE, feature-scattering (FS), and margin losses to create adversarial perturbations and include them for adversarial training to enhance the robustness of the model. We conduct speaker recognition experiments on the Librispeech dataset, and compare the performance with state-of-the-art projected gradient descent (PGD)-based adversarial training which employs only CE objective. The proposed HAT improves adversarial accuracy by absolute 3.29% and 3.18% for PGD and Carlini-Wagner (CW) attacks respectively, while retaining high accuracy on benign examples.

Combination of deep learning and I-vector can significantly improve the performance of speaker recognition system, however, how to optimize the traditional feature parameters and how to model the speaker recognition system through deep learning are two most important research topics. This paper explores the system recognition performance from the types of input and neural network, and researches the optimal feature parameters and the most appropriate neural network structure of a speaker recognition system. Furthermore, the existing speaker recognition algorithms based on deep learning (Deep Neutral Network (DNN), Convolutional Neural Network (CNN)) have been analyzed and several improved models based on deep learning networks have been implemented and compared. Experiment shows that the network model after combination has a higher recognition rate in speaker recognition than the traditional system model.

Speaker recognition is an active research area that contains notable usage in biometric security and authentication system. Currently, there exist many well-performing models in the speaker recognition domain. However, most of the advanced models implement deep learning that requires GPU support for real-time speech recognition, and it is not suitable for low-end devices. In this paper, we propose a lightweight text-independent speaker recognition model based on random forest classifier. It also introduces new features that are used for both speaker verification and identification tasks. The proposed model uses human speech based timbral properties as features that are classified using random forest. Timbre refers to the very basic properties of sound that allow listeners to discriminate among them. The prototype uses seven most actively searched timbre properties, boominess, brightness, depth, hardness, roughness, sharpness, and warmth as features of our speaker recognition model. The experiment is carried out on speaker verification and speaker identification tasks and shows the achievements and drawbacks of the proposed model. In the speaker identification phase, it achieves a maximum accuracy of 78%. On the contrary, in the speaker verification phase, the model maintains an accuracy of 80% having an equal error rate (ERR) of 0.24.

The goal of this paper is to review automatic systems for forensic speaker recognition (FSR) based on scientifically approved methods for calculation and interpretation of biometric evidence. The objective of this paper is not to promote one speaker recognition method against another, but is to make available to the biometric research community data-driven methodology combining automatic speaker recognition techniques and a rigorous forensic experimental background. Forensic speaker recognition is the process of determining if a specific individual (suspected speaker) is the source of a questioned speech recording (trace). This paper aims at reviewing forensic automatic speaker recognition (FASR) methods that provide a coherent way of quantifying and presenting recorded speech as biometric evidence, as well as the assessment of its strength (likelihood ratio) in the Bayesian interpretation framework compatible with interpretations in other forensic disciplines. Forensic speaker recognition has proven an effective tool in the fight against crime, yet there is a constant need for more research due to the difficulties involved because of the within-speaker (within-source) variability, between-speakers (between-sources) variability, and differences in recording sessions conditions.KeywordsLikelihood RatioGaussian Mixture ModelVector QuantizationForensic ScienceSpeaker RecognitionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.