Learning Speaker-Specific Characteristics With a Deep Neural Architecture

Although the field of automatic speaker or speech recognition has been extensively studied over the past decades, the lack of robustness has remained a major challenge. The missing data technique (MDT) is a promising approach. However, its performance depends on the correlation across frequency bands. This paper presents a new reconstruction method for feature enhancement based on the trait. In this paper, the degree of concentration across frequency bands is measured with principal component analysis (PCA). Through theoretical analysis and experimental results, it is found that the correlation of the feature vector extracted from the sub-band (SB) is much stronger than the ones extracted from the full-band (FB). Thus, rather than dealing with the spectral features as a whole, this paper splits full-band into sub-bands and then individually reconstructs spectral features extracted from each SB based on MDT. At the end, those constructed features from all sub-bands will be recombined to yield the conventional mel-frequency cepstral coefficient (MFCC) for recognition experiments. The 2-sub-band reconstruction approach is evaluated in speaker recognition system. The results show that the proposed approach outperforms full-band reconstruction in terms of recognition performance in all noise conditions. Finally, we particularly discuss the optimal selection of frequency division ways for the recognition task. When FB is divided into much more sub-bands, some of the correlations across frequency channels are lost. Consequently, efficient division ways need to be investigated to perform further recognition performance.

ABSTRACT: Speech is one of the most natural and efficient forms of communication among humans, and it is becoming increasingly integral to human-computer interaction in modern applications such as virtual assistants, smart devices, and automated customer service systems. Among the various technologies that leverage speech, speaker recognition stands out as a biometric method that identifies or verifies individuals based on the unique characteristics of their voice. This approach has gained significant traction due to its non-intrusive nature, ease of integration, and applicability across various domains such as security, forensics, and personalised user experiences. In this paper, we introduce a novel speaker recognition system specifically designed to identify speakers based on utterances in the Marathi language, a linguistically rich and widely spoken regional language in India. The system utilises Mel-Frequency Cepstral Coefficients (MFCCS) to extract distinguishing vocal features that closely mimic the human auditory perception. MFCCS are particularly effective in capturing speech’s phonetic and acoustic properties, making them a preferred choice in speech and speaker recognition tasks. Vector quantisation (VQ) is applied to optimise the feature set and reduce computational complexity. VQ compresses the high-dimensional MFCC feature vectors into representative clusters without significantly compromising accuracy, thereby enhancing system efficiency. These features are then processed using hidden Markov Models (HMMS), which are adept at modelling temporal sequences and dynamic variations in speech. HMMS offer a statistical framework that effectively captures the sequential nature of speech patterns, leading to more reliable speaker modelling and recognition.

Automatic speech recognition has progressed considerably in the past several decades for most of the European languages, but still, it is a prominent research area for most of the low-resourced languages. This project presents a study to build an automatic speech recognition (ASR) system using the Kaldi toolkit [1] for the Sinhala language which is one of the low-resourced languages with a large lexical variety. We experiment on different deep neural architectures such as pre-trained DNN, DNN, TDNN, TDNN+LSTM to enhance the acoustic modeling process and each of their performances is investigated in this document. A statistical model of GMM-HMM is also trained on the same dataset keeping it as the baseline model for comparing the effectiveness of a deep learning approach. For the language model, a corpus containing more than 20K sentences taken from UCSC LTRL [2] is used to generate the 220K extended lexicon. The experiments are conducted using a phonetically balanced training dataset consisting of twenty-five hours of speech data collected from fifty females and twenty males and tested on 1.6 hours of speech data. The results obtained show that deep neural network architectures exceed the performance of the statistical baseline model by giving a lower Word Error Rate (WER) of 7.48% on the test data set. The best observed lowest WERs were produced by the TDNN architectures.

With the increasing number and popularity of smart devices in the past few decades, especially those that execute different types of health-related applications, their security is a growing concern. Biometric authentication can be used for device security, and speaker recognition (SR) is one of its elegant forms. In this study, an SR system is developed for the security of smart devices, which identifies a person from the shortest utterance of Arabic language, achieving the highest SR rate, by using the benchmark Linguistic Data Consortium (LDC) Arabic dataset. This study focuses on the use of the consonants and vowels of the Arabic language. It is observed that certain consonants or a fusion of certain consonants in Arabic language helps to ascertain the speaker’s identity efficiently. The shortest utterance that better presents the speaker’s identity by using such consonants is revealed. The system was analyzed by using different numbers of consonants and achieved the maximum (100%) SR rate using a fusion of only three consonants of the Arabic language. These consonants or their combinations can be used to develop text for any speaker-based authentication system.

Significance of analytic phase of speech signals in speaker verification

This paper describes and discusses the "STBU" speaker recognition system, which performed well in the NIST Speaker Recognition Evaluation 2006 (SRE). STBU is a consortium of four partners: Spescom DataVoice (Stellenbosch, South Africa), TNO (Soesterberg, The Netherlands), BUT (Brno, Czech Republic), and the University of Stellenbosch (Stellenbosch, South Africa). The STBU system was a combination of three main kinds of subsystems: 1) GMM, with short-time Mel frequency cepstral coefficient (MFCC) or perceptual linear prediction (PLP) features, 2) Gaussian mixture model-support vector machine (GMM-SVM), using GMM mean supervectors as input to an SVM, and 3) maximum-likelihood linear regression-support vector machine (MLLR-SVM), using MLLR speaker adaptation coefficients derived from an English large vocabulary continuous speech recognition (LVCSR) system. All subsystems made use of supervector subspace channel compensation methods-either eigenchannel adaptation or nuisance attribute projection. We document the design and performance of all subsystems, as well as their fusion and calibration via logistic regression. Finally, we also present a cross-site fusion that was done with several additional systems from other NIST SRE-2006 participants.

Theft of private data became a threat of crime in cyberspace. This issue was in line with rapid development of data technology, especially online transactions. To attenuate this problem, voice biometrics was developed as an answer to keep up security identity. This paper develops the voice biometric framework based on Convolutional Neural Network Depthwise Separable Convolution (DSC) model and the fusion of Discrete Wavelet Transform (DWT) and Mel Frequency Cepstral Coefficients (MFCC). Such a scheme has targeted to increase the high accuracy, to reduce the burden of high computational costs and to speed up the performance of classification process time. We conduct three testing performance, i.e. voice Biometric Training Performance, speaker Recognition Performance (”Who is speaking?”), and Speech Recognition performance (”What keyword is uttered?”). For each of the testing, the results are compared with CNN Standard performance. The training results has shown that CNN DSC model has reduced the amount of training parameters to 364,506, leading to accelerate the performance of training process time to 5.12 minutes. The results of speaker recognition performance has attained the best performance with an accuracy 99.25%, precision 97.14%, recall 98.17% and F1-score 97.28%. The results of speech recognition performance has been able to improve the best performance with accuracy 100%. It can be concluded that CNN DSC with the fusion of DWT- MFCC has outperformed the CNN Standard. The framework can be applied for the identification and verification of user voices accurately, quickly and efficiently for any applications requiring better security performance.

Speaker identification refers to the process of recognizing human voice using artificial intelligence techniques. Speaker identification technologies are widely applied in voice authentication, security and surveillance, electronic voice eavesdropping, and identity verification. In the speaker identification process, extracting discriminative and salient features from speaker utterances is an important task to accurately identify speakers. Various features for speaker identification have been recently proposed by researchers. Most studies on speaker identification have utilized short-time features, such as perceptual linear predictive (PLP) coefficients and Mel frequency cepstral coefficients (MFCC), due to their capability to capture the repetitive nature and efficiency of signals. Various studies have shown the effectiveness of MFCC features in correctly identifying speakers. However, the performances of these features degrade on complex speech datasets, and therefore, these features fail to accurately identify speaker characteristics. To address this problem, this study proposes a novel fusion of MFCC and time-based features (MFCCT), which combines the effectiveness of MFCC and time-domain features to improve the accuracy of text-independent speaker identification (SI) systems. The extracted MFCCT features were fed as input to a deep neural network (DNN) to construct the speaker identification model. Results showed that the proposed MFCCT features coupled with DNN outperformed existing baseline MFCC and time-domain features on the LibriSpeech dataset. In addition, DNN obtained better classification results compared with five machine learning algorithms that were recently utilized in speaker recognition. Moreover, this study evaluated the effectiveness of one-level and two-level classification methods for speaker identification. The experimental results showed that two-level classification presented better results than one-level classification. The proposed features and classification model for identifying a speaker can be widely applied to different types of speaker datasets.

Focused on the issue that the robustness of traditional Mel Frequency Cepstral Coefficients (MFCC) feature degrades drastically in speaker recognition system, a kind algorithm that based improved Gammatone Frequency Cepstral Coefficients (GFCC) is proposed. The different between traditional MFCC and GFCC is that GFCC uses Gammatone filter bank to replace Mel filter bank to improve robustness. On this basis, this paper proposes one way that use Multitaper Estimation, MVA (Mean Subtraction, Variance Normzlization and Autoregressive Moving Average Filter)and other technologies to further enhance its robustness and tested with TIMIT speech database. The experimental results show that under different noise and different SNR, the improved GFCC that proposed by this paper has the lowest equal error rate and the best robustness, especially in the noise ratio is lower than 10dB, has greater advantage compared to other algorithms.

Local spectral variability features for speaker verification

Mismatch between training and testing data is a major error source for both automatic speech recognition (ASR) and automatic speaker identification (ASI). In this paper, we first present a statistical weighting concept to exploit the unequal sensitivity of mel-frequency cepstral coefficients (MFCC) components to against the mismatch, such as ambient noise, recording equipment, transmission channels, and inter-speaker variations. We further design a new Kullback-Leibler (KL) distance based weighting algorithm according to the proposed weighting concept to real-world problems in which the label information is often not provided. We examine our algorithm in ASR with mismatch by different speakers and also in ASI with mismatch by channel noises. Experimental results demonstrate the effectiveness and robustness of our proposed method.

Human speech signal is rich with information such as the identity of the speaker, the spoken message, emotional and physical state of the speaker, the spoken language, gender, age and other information. Automatic Speech Recognition (ASR) involves complex tasks aimed at recognition and translation of human speech to text by computers. Phoneme Recognition means recognizing the phonemes associated with speech utterance and is a part of ASR. Developing a phonetic engine and enhancing its performance can lead to significant improvement in ASR. In this paper we propose Artifical Neural Network (ANN) based Phoneme Modeling. We compare performance of speech features such as Inner Hair Cell Coefficients (IHCC) and Mel-Frequency Cepstral Coefficient (MFCC) using various neural network architectures, different optimization algorithms, change in input data vector dimensionality (corresponding to different contextual information) and also increasing the number of epochs. Experiments were carried out on TIMIT database. Our experimental results indicates MFCC performs much better than IHCC feature and the best optimization algorithm is found to be SGD for IHCC and Adagrad for MFCC features by trying out with different sizes of input vector, changing the number of training iterations, training algorithms for various neural network architectures.

In this study, a hybrid security system is proposed. The proposed system is composed of two subsystems namely iris recognition system (IRS) and speaker recognition system (SRS). Pre-processing, feature extraction and feature matching are the main steps of these systems. In IRS subsystem, Gaussian filter, Canny edge detector, Hough transform, and histogram equalization is performed for pre-processing, respectively. After that, by applying 4-level Discrete Wavelet Transform (DWT) to pure iris image, the iris image is decomposed into four sub-bands (LL4, LH4, HL4 and HH4). In order to extract the feature vector from iris pattern, the LH4, HL4 and HH4 sub-bands (matrices) are merged into one matrix. Finally the matrix is transformed in vector to obtain the feature vector of iris image. For SRS subsystem, the pre-processing step includes spectral arrangement, silence part removing and band limitation operations. After pre-processing, frame blocking and windowing are applied to the long-term speech samples and then Fast Fourier Transform (FFT) is performed for the each short-term speech segments (frames). Finally, the Mel Frequency Cepstral Coefficients (MFCC) technique is performed in order to obtain feature vector of the speech. The feature matching step of both IRS and SRS is implemented with Dynamic Time Warping (DTW) which is an efficient algorithm to measure the distance between two vectors. According to the DTW results, the false acceptance rate (FAR) is zero and false rejecting rate (FRR) is about 4 % for the proposed hybrid system.

The mismatch between the training conditions and the test conditions severely degrades the performance of speaker verification. Aiming at solving this problem, this paper presents a method which uses a combination of improved nonnegative matrix factorization (IMNMF) and mel frequency cepstral coefficients (MFCC) with feature warping for improving identity-vector (i-vector) speaker verification performance in noisy environment. Unlike the traditional nonnegative matrix factorization (NMF), IMNMF uses extra free basis vectors to capture the features which are not included in training data, and linear constraints on dictionary atoms. Feature warping is used to remove channel noises. Therefore, the proposed method can reduce distortion of reconstructed speech while enhancing the recovered speech quality. The performance of i-vector speaker verification is evaluated by using the short utterance database and the NOISEX-92 database. The experiment results indicate that the score level fusion of feature-warped IMNMF-MFCC and feature-warped MFCC is superior to the baseline system at the equal error rate under the majority of signal-to-noise ratios (SNRs).

Speaker recognition systems mostly employ mel frequency cepstral coefficients (MFCC). Performance of these systems is generally affected by background noise, transmission medium, etc. Further, they do not perform well in text‐independent environment with limited training data. For enhanced performance, the set of parameters used should separate the speaker‐dependent information from the linguistic information. Towards this end, application of parameters of the harmonic plus noise model (HNM)‐based analysis is investigated. HNM divides the speech spectrum into harmonic and noise bands separated by a dynamically varying maximum voiced frequency. This frequency is a speaker‐dependent parameter and its estimation is not affected by moderate SNR degradation. A speaker recognition system was devised using three HNM parameters, namely, maximum voiced frequency, relative noise band energy, and pitch. It gave‐performed comparably to that of MFCC‐based recognition, for a group of ten speakers. An enhanced performan...