Effects of language mismatch in automatic forensic voice comparison using deep learning embeddings.

Language mismatching is considered one of the biggest challenges in achieving adequate speaker verification. The number of bilingual speakers worldwide is increasing, making speaker verification for speech technology more challenging. The main objective of this study is to examine the effect of language mismatch between training and test conditions on the performance of the speaker verification model, with a specific focus on children's speech by utilizing pre-trained and fine-tuned deep speaker embedding models and by investigating two child speech datasets (Samromur and kidsTALC). This work utilises two time-delay neural networks (TDNN): X-Vector and Emphasized Channel Attention, Propagation and Aggregation (ECAPA) to extract the embedding features of speech samples. For evaluation of the performance of speaker verification, we used the likelihood-ratio framework by using the likelihood-ratio score calculation method based on children’s voices and employing measures such as log-likelihood ratio cost (Cllr) and equal error rate (EER). The experimental results indicate that the language variety between training and testing utterances significantly degrades speaker verification performance compared to multi-language training; however, fine-tuning still performs better than pretrained models by 11.5% in the best performance of ECAPA-TDNN.

Phonetic information is one of the most essential components of a speech signal, playing an important role for many speech processing tasks. However, it is difficult to integrate phonetic information into speaker verification systems since it occurs primarily at the frame level while speaker characteristics typically reside at the segment level. In deep neural network-based speaker verification, existing methods only apply phonetic information to the frame-wise trained speaker embeddings. To improve this weakness, this paper proposes phonetic adaptation and hybrid multi-task learning and further combines these into c-vector and simplified c-vector architectures. Experiments on National Institute of Standards and Technology (NIST) speaker recognition evaluation (SRE) 2010 show that the four proposed speaker embeddings achieve better performance than the baseline. The c-vector system performs the best, providing over 30% and 15% relative improvements in equal error rate (EER) for the core-extended and 10 s–10 s conditions, respectively. On the NIST SRE 2016, 2018, and VoxCeleb datasets, the proposed c-vector approach improves the performance even when there is a language mismatch within the training sets or between the training and evaluation sets. Extensive experimental results demonstrate the effectiveness and robustness of the proposed methods.

The purpose of this paper is to preliminarily present the status of the paradigm shift or new paradigm in forensic voice comparison in Peru. First, the ambiguous terminology (such as forensic acoustics, voice identification or forensic phonoaudiology as forensic voice comparison) used to avoid its inappropriate use in academic and judicial situations is reviewed. Second, forensic phonetics and its areas of practice are presented in general terms. Third, the methodological approaches in voice comparison and the way of expressing the results are presented and it is explained why the forensic expert in voice comparison should opt for the analysis and results through the likelihood ratio. Fourth, the basis of the paradigm shift and its influence on forensic voice comparison is discussed in depth. Fifth, the current state of research in forensic voice comparison in Peru is presented. As a result of this research, it can be argued that the current studies demonstrate a transition towards a paradigm shift, especially through the use of automated programs.

This paper proposes a novel Sequence-to-Sequence Neural Diarization (S2SND) framework to perform online and offline speaker diarization. It is developed from the sequence-to-sequence architecture of our previous target-speaker voice activity detection system and then evolves into a new diarization paradigm by addressing two critical problems. 1) Speaker Detection: The proposed approach can utilize partially given speaker embeddings to discover the unknown speaker and predict the target voice activities in the audio signal. It does not require a prior diarization system for speaker enrollment in advance. 2) Speaker Representation: The proposed approach can adopt the predicted voice activities as reference information to extract speaker embeddings from the audio signal simultaneously. The representation space of speaker embedding is jointly learned within the whole diarization network without using an extra speaker embedding model. During inference, the S2SND framework can process long audio recordings blockwise. The detection module utilizes the previously obtained speaker-embedding buffer to predict both enrolled and unknown speakers' voice activities for each coming audio block. Next, the speaker-embedding buffer is updated according to the predictions of the representation module. Assuming that up to one new speaker may appear in a small block shift, our model iteratively predicts the results of each block and extracts target embeddings for the subsequent blocks until the signal ends. Finally, the last speaker-embedding buffer can re-score the entire audio, achieving highly accurate diarization performance as an offline system. Experimental results show that our proposed S2SND framework achieves new state-of-the-art diarization error rates (DERs) for online inference on the DIHARD-II (24.41%) and DIHARD-III (17.12%) evaluation sets without using oracle voice activity detection. At the same time, it also refreshes the state-of-the-art performance for offline inference on these benchmarks, with DERs of 21.95% and 15.13%, respectively.

Over the past few years, Large Language Models of Code (Code LLMs) have started to have a significant impact on programming practice. Code LLMs are also emerging as building blocks for research in programming languages and software engineering. However, the quality of code produced by a Code LLM varies significantly by programming language. Code LLMs produce impressive results on high-resource programming languages that are well represented in their training data (e.g., Java, Python, or JavaScript), but struggle with low-resource languages that have limited training data available (e.g., OCaml, Racket, and several others). This paper presents an effective approach for boosting the performance of Code LLMs on low-resource languages using semi-synthetic data. Our approach, called M ulti PL-T, generates high-quality datasets for low-resource languages, which can then be used to fine-tune any pretrained Code LLM. M ulti PL-T translates training data from high-resource languages into training data for low-resource languages in the following way. 1) We use a Code LLM to synthesize unit tests for commented code from a high-resource source language, filtering out faulty tests and code with low test coverage. 2) We use a Code LLM to translate the code from the high-resource source language to a target low-resource language. This gives us a corpus of candidate training data in the target language, but many of these translations are wrong. 3) We use a lightweight compiler to compile the test cases generated in (1) from the source language to the target language, which allows us to filter our obviously wrong translations. The result is a training corpus in the target low-resource language where all items have been validated with test cases. We apply this approach to generate tens of thousands of new, validated training items for five low-resource languages: Julia, Lua, OCaml, R, and Racket, using Python as the source high-resource language. Furthermore, we use an open Code LLM (StarCoderBase) with open training data (The Stack), which allows us to decontaminate benchmarks, train models without violating licenses, and run experiments that could not otherwise be done. Using datasets generated with M ulti PL-T, we present fine-tuned versions of StarCoderBase and Code Llama for Julia, Lua, OCaml, R, and Racket that outperform other fine-tunes of these base models on the natural language to code task. We also present Racket fine-tunes for two very recent models, DeepSeek Coder and StarCoder2, to show that M ulti PL-T continues to outperform other fine-tuning approaches for low-resource languages. The M ulti PL-T approach is easy to apply to new languages, and is significantly more efficient and effective than alternatives such as training longer.

LSTM-based speaker verification usually uses a fixed-length local segment randomly truncated from an utterance to learn the utterance-level speaker embedding, while using the average embedding of all segments of a test utterance to verify the speaker, which results in a critical mismatch between testing and training. This mismatch degrades the performance of speaker verification, especially when the durations of training and testing utterances are very different. To alleviate this issue, we propose the deep segment attentive embedding method to learn the unified speaker embeddings for utterances of variable duration. Each utterance is segmented by a sliding window and LSTM is used to extract the embedding of each segment. Instead of only using one local segment, we use the whole utterance to learn the utterance-level embedding by applying an attentive pooling to the embeddings of all segments. Moreover, the similarity loss of segment-level embeddings is introduced to guide the segment attention to focus on the segments with more speaker discriminations, and jointly optimized with the similarity loss of utterance-level embeddings. Systematic experiments on Tongdun and VoxCeleb show that the proposed method significantly improves robustness of duration variant and achieves the relative Equal Error Rate reduction of 50% and 11.54% , respectively.

The extraction of speaker-related features through speaker representations or utterance embeddings has been extensively studied for years. Convolutional Neural Networks (CNNs) are the most commonly used backbone network for feature extraction, and deep Residual Networks (ResNets), which incorporate residual connectivity into CNNs, achieve effective frame-level feature extraction inside the same input samples. ResNets have demonstrated impressive performance as speaker embedding extractors. With emerging this backbone and its advanced extensions, such as Res2Net, as dominant backbone networks in speaker recognition, and despite its widespread adoption, a significant gap in current speaker embeddings is the unmodified use of these successful backbones from the visual domain, without considering the specific adaptation requirements for the speech field. Tailoring the backbone to account for the unique characteristics of speech representations ensures effective learning in Automatic Speaker Verification (ASV) systems. This thesis concentrates on the backbone network, the fundamental and most critical component of Deep Neural Networks (DNNs), responsible for extracting features from input data and mapping them into distinct feature representations. One of the key questions addressed in the thesis is the following: Which backbone is most robust and yields the most generalized speaker representation, with the potential to be applied to many existing ASV systems?. Through a strategic evaluation, the study analyzed the performance of integrating different residual-based backbones into the most popular ResNet-based fundamental baselines, ResNet-34 and ECAPA-TDNN based models, with highly different model complexity and CNN stem representations. To this end, the proposed architectures were evaluated on the publicly available VoxCeleb dataset. This thesis introduces an innovative approach to speaker modeling by incorporating scale and cardinality dimensions for the first time. Experimental findings reveal that utilizing multi-scale and multi-branch aggregated residual transformations within the Res2NeXt backbone yields considerable performance improvements. Notably, 6s8g and 8s8g variants of Res2NeXt, prove particularly beneficial in improving the representational capability of learned features for speaker embedding. The consistent superiority, excellent compatibility, and robustness of Res2NeXt compared to the current popular ResNet and Res2Net backbones across various architectures and conditions demonstrate its broad applicability and effectiveness as a backbone in speaker modeling. To further evaluate the Robustness and Generalization (R&G) capabilities of the proposed speaker embeddings, the thesis expanded its scope to cross-domain applications, specifically Speech Emotion Recognition (SER), one of the most challenging downstream tasks. A comparative analysis of the proposed speaker embeddings was conducted using two benchmark datasets, IEMOCAP and CREMA-D. The overarching goal of the thesis is to contribute to the development of general-purpose and universal speech representations that generalize effectively across multiple speech-related tasks. By examining the alignment and divergence of embeddings for ASV and SER, the study seeks to answer a fundamental question: Can a single representation framework cater to both speaker verification and emotion recognition or are task-specific embeddings inevitable?. The results confirm that speaker embeddings derived from the Res2NeXt backbone are also highly effective for the SER task, maintaining their performance across diverse evaluation protocols and data variations. This demonstrates the generalizability of Res2NeXt embeddings in the cross-domain task. The study attributes their success to the ability of Res2NeXt-based models to capture rich speech features, including tone, pitch, and phonation patterns, during speaker recognition training—features that are also vital for emotion recognition.

Speaker verification (SV) aims to determine whether the speaker's identity of a test utterance is the same as the reference speech. In the past few years, extracting speaker embeddings using deep neural networks for SV systems has gone mainstream. Recently, different attention mechanisms and Transformer networks have been explored widely in SV fields. However, utilizing the original Transformer in SV directly may have frame-level information waste on output features, which could lead to restrictions on capacity and discrimination of speaker embeddings. Therefore, we propose an approach to derive utterance-level speaker embeddings via a Transformer architecture that uses a novel loss function named diffluence loss to integrate the feature information of different Transformer layers. Therein, the diffluence loss aims to aggregate frame-level features into an utterance-level representation, and it could be integrated into the Transformer expediently. Besides, we also introduce a learnable mel-fbank energy feature extractor named time-domain feature extractor that computes the mel-fbank features more precisely and efficiently than the standard mel-fbank extractor. Combining Diffluence loss and Time-domain feature extractor, we propose a novel Transformer-based time-domain SV model (DT-SV) with faster training speed and higher accuracy. Experiments indicate that our proposed model can achieve better performance in comparison with other models.

As part of the Expert Evidence series this chapter is intended to be accessible to lawyers, judges, police officers, and potential jury members; however, it is hoped that this chapter will also be of interest to forensic scientists, phoneticians / speech scientists, speech-processing engineers, and students of all these disciplines. It introduces forensic speech science in a relatively non-technical way, assuming a reader who has no prior knowledge of the subject. The 2010 edition had a heavy focus on acoustic-phonetic statistical approaches to forensic voice comparison. The 2018 edition has a heavier focus on automatic approaches. The 2018 edition includes examples of forensic voice comparison based on real cases. The 2018 edition also includes expended coverage of other branches of forensic speech science. Topics covered include: The likelihood ratio framework for the evaluation of forensic evidence. Approaches to forensic voice comparison. Assessing the validity and reliability of forensic-comparison systems. Speaker identification by laypeople. Disputed utterance analysis.

Different types of documents, such as financial, commercial, security, certificate, and judicial documents, require signatures for verification. However, conventional signature verification processes face difficulties with image-based verification due to the rising volume of documents and digital transactions. The transition to digital platforms requires effective verification systems capable of handling real-world complexities. The current study employs machine learning and deep learning techniques, including Support Vector Machine (SVM) classification and ResNet-50 for feature extraction, to address these challenges. By reducing intra-class variability and increasing inter-class separability, Triplet Loss optimization enhances the model’s ability to learn discriminative features. Moreover, Error Level Analysis (ELA) is used to detect pixel inconsistencies and improve feature quality. The Adam optimizer is applied to accelerate convergence and improve training stability. In addition, hyperparameter tuning is performed to enhance SVM performance. The efficiency of the proposed model is evaluated using the ICDAR-2011 dataset to validate the effectiveness of this hybrid approach. The results demonstrate that combining Triplet Loss and ELA preprocessing with ResNet-50 embeddings for SVM classification significantly improves signature verification accuracy. In this study, a pretrained model is utilized for offline signature verification. Type I error (False Acceptance Rate, FAR) occurs when a forged signature is incorrectly accepted as genuine. Conversely, Type II error (False Rejection Rate, FRR) occurs when a genuine signature is incorrectly rejected as forged. The proposed architecture employs ResNet-50 for feature extraction and a Siamese network for classification. This combination achieves a high accuracy of 98.81%.

Forensic voice comparison (FVC) is gaining global recognition as a scientific method for speaker identification in legal proceedings. In Indonesia, however, the application of FVC remains underdeveloped, despite increasing reliance on digital and audio based evidence in criminal cases. This study presents a legally and methodologically robust framework for implementing FVC within the Indonesian judicial context. The research integrates Indonesian legal standards (KUHAP Articles 183–184, UU ITE) with international forensic protocols, including ISO/IEC 27037 for digital evidence handling. A combination of acoustic features MFCCs, F0, formants, and VTLN was extracted from disputed and reference voice samples. Likelihood ratios (LRs) were calculated using Gaussian Mixture Models, with score calibration via logistic regression. Results showed mean log LR values of 2.1 for genuine trials and –1.8 for impostor trials, with an ROC AUC of 0.91. Visual tools, including Tippett plots and ROC curves, were used to interpret and communicate evidence reliability. The findings confirm that calibrated, probabilistic FVC methods are feasible and legally admissible in Indonesia. However, challenges remain in handling low quality recordings, maintaining chain of custody, and bridging communication gaps between scientific experts and legal practitioners. The study recommends structured training, standardized protocols, and the use of visual aids to enhance evidentiary transparency. This framework lays the foundation for a scalable, court ready FVC system aligned with national law and global best practices. It supports interdisciplinary cooperation aimed at strengthening Indonesia’s forensic infrastructure.

Multi-Language Speech-to-Text Translation (ST) plays a pivotal role in bridging linguistic barriers by converting spoken language into written text across different languages.This project aims to develop a robust ST model tailored for low-resource Indian languages, specifically targeting the Indo-Aryan and Dravidian language families.The dataset used consists of speeches from conferences and TED Talks, along with their corresponding transcriptions in English (source language) and translations in Hindi, Bengali, and Tamil (target languages).By tackling the lack of data and disparities in attention within low-resource languages, the paper strives to create an efficient ST system capable of real-world deployment.Additionally, existing resources in related languages are leveraged and word-level translation resources are explored to enhance translation accuracy.

This study evaluates the dual role of translators and interpreters working for NGOs involved in the health sector in developing countries. Given the crucial importance of public health in a nation’s development, we assess the way grassroots participation in healthcare projects and activities is enhanced through the translation of healthcare information into low-resource languages. This study, epistemologically grounded in the theory of planned behaviour (TPB), adopts a qualitative and quantitative methodological approach to correlate the TPB and the target-language choices of translator-interpreters and the translation strategies they adopt working with low-resource languages. The survey responses and audio recordings analysed are those of translators or interpreters working in a rural context in the Anglophone regions of Cameroon. The target language in question is Pidgin English, a low-resource oral language. The evidence gathered leads to the finding that various de-terminologization strategies are employed to ensure that messages are adequately disseminated and that the local community members are involved in healthcare activities. The study seeks to bring together translation, interpreting, the TPB, and international development in which the involvement of NGOs underscores North–South cooperation in the strategic domain of global healthcare.

Refining the relevant population in forensic voice comparison – A response to Hicks et alii (2015) The importance of distinguishing information from evidence/observations when formulating propositions

Introduction: Author attribution is a critical task in the domain of computational linguistics, particularly when dealing with low-resourced languages. These languages often lack sufficient annotated datasets and robust linguistic tools, making natural language processing (NLP) applications challenging. A piece of text generally reflects the unique stylistic traits of its author, including their use of vocabulary, punctuation, and sentence structures. Identifying the correct author based on these textual cues is the central aim of author attribution. Despite its importance, this area remains relatively unexplored for low-resourced languages due to the inherent data and resource limitations. Objectives: This study aims to address the problem of author attribution in low-resourced languages by leveraging deep learning techniques. Specifically, the objectives are to investigate how neural network models like Recurrent Neural Networks (RNN) and hybrid Convolutional Neural Network with Long Short-Term Memory (CNN-LSTM) can effectively capture stylistic features of text, and to design an ensemble model that combines these approaches for enhanced performance in multi-author scenarios. Methods: To achieve the stated objectives, we employed RNN and CNN-LSTM architectures to model the stylistic nuances present in textual data. These models were trained and tested on the AAALC dataset, which contains writings from multiple authors in a low-resourced language. The performance of each model was evaluated using key classification metrics such as accuracy and F1-score. Additionally, an ensemble model combining the outputs of RNN and CNN-LSTM was proposed to further boost classification performance by leveraging the strengths of both architectures. Results: The results of our experiments demonstrated that deep learning models are highly effective in the context of author attribution for low-resourced languages. Among the evaluated models, the ensemble approach achieved the best results, with an accuracy of 84.38% and an F1-score of 79.0%. These results indicate that combining neural network architectures can significantly improve classification accuracy by capturing a richer representation of stylistic features. Conclusions: In conclusion, this study validates the potential of neural network-based models for tackling the author attribution task in low-resourced languages. The proposed ensemble model not only achieved strong performance metrics but also demonstrated the practical viability of deep learning approaches in linguistic contexts where traditional resources are scarce. These findings contribute to the advancement of inclusive NLP systems and offer valuable insights for further research in multilingual and resource-constrained settings.