Language Identification in Overlapped Multi-lingual Speeches

In recent years, with the rise of human-computer interaction and the successful application of end-to-end models in the field of speech recognition, the construction of end-to-end speech recognition models has received extensive attention. Relying on the multi-task learning method and the connection between language identification and speech recognition, we proposed an end-to-end Transformer model, which is a multilingual speech recognition model integrating language identification. The model takes the speech recognition task as the main task and the language identification task as the auxiliary task. In this paper, the validity of the model is verified by using the datasets of 13 languages in the 2021 Oriental Language Recognition challenge (OLR). The experimental results show that the model constructed in this paper has a relative improvement of 37.46% in the speech recognition task compared with the baseline system proposed by the OLR organizer. The accuracy of language identification reaches 89.70 %. The results can get the fifth place in the 2021 OLR constraint track of speech recognition equally.

Speaker and language recognition in noisy and degraded channel conditions continue to be a challenging problem mainly due to the mismatch between clean training and noisy test conditions. In the presence of noise, the most reliable portions of the signal are the high energy regions which can be used for robust feature extraction. In this paper, we propose a front end processing scheme based on autoregressive (AR) models that represent the high energy regions with good accuracy followed by a modulation filtering process. The AR model of the spectrogram is derived using two separable time and frequency AR transforms. The first AR model (temporal AR model) of the sub-band Hilbert envelopes is derived using frequency domain linear prediction (FDLP). This is followed by a spectral AR model applied on the FDLP envelopes. The output 2-D AR model represents a low-pass modulation filtered spectrogram of the speech signal. The band-pass modulation filtered spectrograms can further be derived by dividing two AR models with different model orders (cut-off frequencies). The modulation filtered spectrograms are converted to cepstral coefficients and are used for a speaker recognition task in noisy and reverberant conditions. Various speaker recognition experiments are performed with clean and noisy versions of the NIST-2010 speaker recognition evaluation (SRE) database using the state-of-the-art speaker recognition system. In these experiments, the proposed front-end analysis provides substantial improvements (relative improvements of up to 25%) compared to baseline techniques. Furthermore, we also illustrate the generalizability of the proposed methods using language identification (LID) experiments on highly degraded high-frequency (HF) radio channels and speech recognition experiments on noisy data.

This study proposes a novel deep neural network (DNN) based approach to language identification (LID) for the NIST 2015 Language Recognition (LRE) i-Vector Machine Learning Challenge. State-of-the-art DNN based LID systems utilize large amounts of labeled training data. The 2015 LRE i-Vector Machine Learning Challenge limits the access to only ready-to-use i-Vectors for LID system training and testing. This poses unique challenges in designing DNN based LID systems, since optimized front-ends and network architectures can no longer be used. We propose to use the training i-Vectors to train an initial DNN for LID. Next, we present a novel strategy to use this initial DNN to estimate out-of-set language labels from the development data. The final DNN for LID is trained using the original training data, and the estimated out-of-set language data. We show that augmenting the training set with out-of-set labels leads to significant improvement in the LID performance. Our approach obtains very competitive costs (defined by NIST) of 26.56, and 25.98 respectively, on the progress and evaluation subsets of the challenge. Since the amount of training data is very limited (300 i-Vectors per language), this study outlines a successful recipe for DNN based LID using very limited resources.

Adapting multilingual automatic speech recognition (MASR) models to support new languages is crucial for enhancing global communication accessibility. However, extending these models often leads to catastrophic forgetting, impairing their ability to accurately process previously learned languages. To address this, we present a mixture-of-experts (MoE) strategy that employs Low-Rank Adaptation (LoRA) experts, each dedicated to a specific language. Unlike previous approaches that rely on a small neural network for gating, we utilize the language identification (LID) outputs of the MASR model itself to more precisely activate the appropriate LoRA experts. Our method involves training two sets of LoRAs for LID and ASR, respectively: the LID LoRAs adapt the MASR model for LID across both existing and new languages, and the ASR LoRAs consist of multiple language-specific LoRA experts for ASR. During inference, we first employ the LID LoRAs to identify the language, followed by the activation of the corresponding ASR LoRAs based on this identification. Unfortunately, this approach means LID errors also cause incorrect ASR LoRAs activation. We perform language-wise beam search to allow self-correction for such mistakes. When applied to the Whisper model and integrated with ten new languages from the Common Voice dataset, our approach achieves up to 13.8% and 10.4% relative improvements in word error rate (WER) for new and previously learned languages, respectively. This method effectively reduces catastrophic forgetting with less than 3% additional computation overhead compared to the standard LoRA implementation.

Objective: Phones are basic sound units available in the spoken data. Languages differ among themselves due to use of different phone sets. This paper analyzes some aspects of automatic phone recognition and subsequent identification of a few languages from North Eastern region of India using Phonetic Engine (PE). Methods and Statistical Analysis: PE is a system which converts a speech sample into some symbolic form so that these symbols are capable of capturing all the information carried by the speech sample. In the development of PEs, the International Phonetic Alphabet (IPA) symbols are used in the data transcription process. In modelling the phonetic units Hidden Markov Models (HMM) have been used in the training phase. These trained HMMs are then used in phone recognitions leading to the identification of language(s) of unknown test utterances. Findings: PEs are built for three Indian Languages and one dialect, namely Manipuri, Assamese and Bengali and the Kakching dialect of Manipuri. These languages are widely spoken across the North Eastern region of India. The overall phone recognition accuracies reported by the PEs for the above selected languages are 62:11% for standard Manipuri language, 59:0% for Kakching dialect of Manipuri, 43:28% for standard Assamese and 48:58% for Bengali language. Application: Automatic LID is possible using a set of PEs in testing unknown utterances due to the language bias of these systems. Various level of identification rates reported in some LID tasks carried out with PEs are discussed here to make an analysis of the issues belonging to it.

Gaussian mixture model is an effective method for speaker-independent language identification tasks. Gaussian mixture bigram model integrates bigram time correlation to extend the GMM. A language identification algorithm of GMBM-UBBM is proposed based on GMBM and GMM-UBM and some experiments are conducted using OGI-TS multilanguage telephone speech corpus. Simulation results demonstrate the effectiveness of GMBM-UBBM for language identification tasks and use of this model allows the proposed system to distinguish between the three languages with maximal 4.378% improvement accuracy superior to conventional GMM-UBM.

In many inference and learning tasks, collecting large amounts of labeled training data is time consuming and expensive, and oftentimes impractical. Thus, being able to efficiently use small amounts of labeled data with an abundance of unlabeled data—the topic of semi-supervised learning (SSL) [1]—has garnered much attention. In this paper, we look at the problem of choosing these small amounts of labeled data, the first step in a bootstrapping paradigm. Contrary to traditional active learning where an initial trained model is employed to select the unlabeled data points which would be most informative if labeled, our selection has to be done in an unsupervised way, as we do not even have labeled data to train an initial model. We propose using unsupervised clustering algorithms, in particular integrated sensing and processing decision trees (ISPDTs) [2], to select small amounts of data to label and subsequently use in SSL (e.g. transductive SVMs). In a language identification task on the CallFriend1 and 2003 NIST Language Recognition Evaluation corpora [3], we demonstrate that the proposed method results in significantly improved performance over random selection of equivalently sized training data.

This work deals with front end processing of co-channel speaker identification problem. A co-channel speech consists of mixture of speech signals from two speakers and one microphone. Most of the methods on speaker identification work well for noise free and single speaker environment and the performance reduces under noisy and multi speaker environment. We propose EMD (Empirical mode decomposition) based denoising at the front end along with pitch estimation to separate the co-channel speech mixture. Results show that speaker id correct rate increases after denoising and separating the speakers from the mixture using proposed method at the front end of speaker identification.

The development of Multilingual Large Vocabulary Continuous Speech Recognition systems involves issues as: Language Identification, Acoustic-Phonetic Decoding, Language Modelling or the development of appropriated Language Resources. The interest on Multilingual Systems arouses because there are three official languages in the Basque Country (Basque, Spanish, and French), and there is much linguistic interaction among them, even if Basque has very different roots than the other two languages. This paper describes the development of a Language Identification (LID) system oriented to robust Multilingual Speech Recognition for the Basque context. The work presents hybrid strategies for LID, based on the selection of system elements by Support Vector Machines and Multilayer Perceptron classifiers and stochastic methods for speech recognition tasks (Hidden Markov Models and n-grams).KeywordsAutomatic Speech RecognitionMultilingual SystemLarge Vocabulary Continuous Speech RecognitionUnder-resourced LanguagesCross-lingual strategies

Forensic speaker recognition: A new method based on extracting accent and language information from short utterances

Unsupervised language identification based on Latent Dirichlet Allocation

The introduction of a phonetic engine in the literature is that it is a system which transforms a speech signal into some symbolic form. Phone recognition is a primary task of a PE. Among the various other applications of PE, one very useful application is the Language Identification (LID). This chapter discusses some issues pertaining to the use of PE in phone recognition as well as in language identification. Here, we have used PEs for three Indian Languages: Manipuri, Assamese and Bengali, in building the LID system. These languages are widely spoken in the Northeastern region of India. In the development of PEs, the International Phonetic Alphabet (IPA) symbols are used in the data transcription process. In modeling the phonetic units Hidden Markov Models (HMM) have been used in the training phase. These trained HMMs are then used in phone recognitions leading to the identification of language(s) of unknown test utterances. The overall phone recognition accuracies reported by the existing PEs for the above selected languages are \(62.11\%\) for standard Manipuri language, \(59.0\%\) for Kakching Dialect of Manipuri, \(43.28\%\) for standard Assamese and \(48.58\%\) for Bengali language. Automatic LID is possible using a set of PEs in testing unknown utterances due to the language bias of these systems. Various levels of identification rates reported in some LID tasks carried out with PEs are discussed here to look into the issues belonging to it.

Language Identification of Intra-Word Code-Switching for Arabic–English

Speaker recognition is an active research area that contains notable usage in biometric security and authentication system. Currently, there exist many well-performing models in the speaker recognition domain. However, most of the advanced models implement deep learning that requires GPU support for real-time speech recognition, and it is not suitable for low-end devices. In this paper, we propose a lightweight text-independent speaker recognition model based on random forest classifier. It also introduces new features that are used for both speaker verification and identification tasks. The proposed model uses human speech based timbral properties as features that are classified using random forest. Timbre refers to the very basic properties of sound that allow listeners to discriminate among them. The prototype uses seven most actively searched timbre properties, boominess, brightness, depth, hardness, roughness, sharpness, and warmth as features of our speaker recognition model. The experiment is carried out on speaker verification and speaker identification tasks and shows the achievements and drawbacks of the proposed model. In the speaker identification phase, it achieves a maximum accuracy of 78%. On the contrary, in the speaker verification phase, the model maintains an accuracy of 80% having an equal error rate (ERR) of 0.24.

This paper presents a number of experiments describing the use of machine learning algorithms and bag-of-words to the task of automatic language identification. The paper focuses on the identification of language varieties, which is a known weakness of general purpose language identification methods. This question was addressed by a number of studies in the recent years, most of them relying on character n-gram language models. In this paper, I experiment simple bag-of-words and compare the results with previously proposed n-gram-based approaches. To perform these classification experiments three algorithms were used: Multinomial Naive Bayes (MNB), Support Vector Machines (SVM) and the J48 classifier.