A Machine of Few Words: Interactive Speaker Recognition with Reinforcement Learning

<span lang="EN-US">Current Automatic Speaker Recognition (ASR) System has emerged as an important medium of confirmation of identity in many businesses, ecommerce applications, forensics and law enforcement as well. Specialists trained in criminological recognition can play out this undertaking far superior by looking at an arrangement of acoustic, prosodic, and semantic attributes which has been referred to as structured listening. An algorithmbased system has been developed in the recognition of forensic speakers by physics scientists and forensic linguists to reduce the probability of a contextual bias or pre-centric understanding of a reference model with the validity of an unknown audio sample and any suspicious individual. Many researchers are continuing to develop automatic algorithms in signal processing and machine learning so that improving performance can effectively introduce the speaker’s identity, where the automatic system performs equally with the human audience. In this paper, I examine the literature about the identification of speakers by machines and humans, emphasizing the key technical speaker pattern emerging for the automatic technology in the last decade. I focus on many aspects of automatic speaker recognition (ASR) systems, including speaker-specific features, speaker models, standard assessment data sets, and performance metrics</span>

Speech feature extraction is the most significant step in any Automatic speaker recognition system. In the last 60 years a lot of research has gone into parametric representation of these speech features. Several techniques are currently being used for Automatic Speaker Recognition. Yet Automatic Speaker Recognition still remains a confront mainly due to variations in speaker's vocal tract with time and health, varying environmental conditions, disparities in the behavior and quality of speech recorders etc. MFCC is a extensively used technique in Automatic speaker recognition. In this paper the performance of MFCC technique was evaluated in a quiet environment. A speaker database containing 30 male and 30 female speakers was created. Two separate experiments were conducted for the performance evaluation of MFCC technique when applied to K means clustering. In the first case the speech features were directly matched. In the second case a VQ codebook was created by clustering the training features of these 60 speakers. A distortion easure based on the minimum Euclidean distance was used for speaker recognition. The failure rate of speaker recognition in first ase was found to be was found to be 10% while in the second case as found to be 14%. Matlab-7.10.0 was used for this study.

ABSTRACT: Speech is one of the most natural and efficient forms of communication among humans, and it is becoming increasingly integral to human-computer interaction in modern applications such as virtual assistants, smart devices, and automated customer service systems. Among the various technologies that leverage speech, speaker recognition stands out as a biometric method that identifies or verifies individuals based on the unique characteristics of their voice. This approach has gained significant traction due to its non-intrusive nature, ease of integration, and applicability across various domains such as security, forensics, and personalised user experiences. In this paper, we introduce a novel speaker recognition system specifically designed to identify speakers based on utterances in the Marathi language, a linguistically rich and widely spoken regional language in India. The system utilises Mel-Frequency Cepstral Coefficients (MFCCS) to extract distinguishing vocal features that closely mimic the human auditory perception. MFCCS are particularly effective in capturing speech’s phonetic and acoustic properties, making them a preferred choice in speech and speaker recognition tasks. Vector quantisation (VQ) is applied to optimise the feature set and reduce computational complexity. VQ compresses the high-dimensional MFCC feature vectors into representative clusters without significantly compromising accuracy, thereby enhancing system efficiency. These features are then processed using hidden Markov Models (HMMS), which are adept at modelling temporal sequences and dynamic variations in speech. HMMS offer a statistical framework that effectively captures the sequential nature of speech patterns, leading to more reliable speaker modelling and recognition.

Speaker and language recognition in noisy and degraded channel conditions continue to be a challenging problem mainly due to the mismatch between clean training and noisy test conditions. In the presence of noise, the most reliable portions of the signal are the high energy regions which can be used for robust feature extraction. In this paper, we propose a front end processing scheme based on autoregressive (AR) models that represent the high energy regions with good accuracy followed by a modulation filtering process. The AR model of the spectrogram is derived using two separable time and frequency AR transforms. The first AR model (temporal AR model) of the sub-band Hilbert envelopes is derived using frequency domain linear prediction (FDLP). This is followed by a spectral AR model applied on the FDLP envelopes. The output 2-D AR model represents a low-pass modulation filtered spectrogram of the speech signal. The band-pass modulation filtered spectrograms can further be derived by dividing two AR models with different model orders (cut-off frequencies). The modulation filtered spectrograms are converted to cepstral coefficients and are used for a speaker recognition task in noisy and reverberant conditions. Various speaker recognition experiments are performed with clean and noisy versions of the NIST-2010 speaker recognition evaluation (SRE) database using the state-of-the-art speaker recognition system. In these experiments, the proposed front-end analysis provides substantial improvements (relative improvements of up to 25%) compared to baseline techniques. Furthermore, we also illustrate the generalizability of the proposed methods using language identification (LID) experiments on highly degraded high-frequency (HF) radio channels and speech recognition experiments on noisy data.

The spoofing countermeasure (CM) systems in automatic speaker verification (ASV) are not typically used in isolation of each other. These systems can be combined, for example, into a cascaded system where CM produces first a decision whether the input is synthetic or bona fide speech. In case the CM decides it is a bona fide sample, then the ASV system will consider it for speaker verification. End users of the system are not interested in the performance of the individual sub-modules, but instead are interested in the performance of the combined system. Such combination can be evaluated with tandem detection cost function (t-DCF) measure, yet the individual components are trained separately from each other using their own performance metrics. In this work we study training the ASV and CM components together for a better t-DCF measure by using reinforcement learning. We demonstrate that such training procedure indeed is able to improve the performance of the combined system, and does so with more reliable results than with the standard supervised learning techniques we compare against.

Acoustical communication is one of the fundamental prerequisites for the existence of human society. Textual language has become extremely important in modern life, but speech has dimensions of richness that text cannot approximate. From speech alone, fairly accurate guesses can be made as to whether the speaker is male or female, adult or child. In addition, experts can extract from speech information regarding e.g. the speaker’s state of mind. As computer power increased and knowledge about speech signals improved, research of speech processing became aimed at automated systems for many purposes. Speaker recognition is the complement of speech recognition. Both techniques use similar methods of speech signal processing. In automatic speech recognition, the speech processing approach tries to extract linguistic information from the speech signal to the exclusion of personal information. Conversely, speaker recognition is focused on the characteristics unique to the individual, disregarding the current word spoken. The uniqueness of an individual’s voice is a consequence of both the physical features of the person vocal tract and the person mental ability to control the muscles in the vocal tract. An ideal speaker recognition system would use only physical features to characterize speakers, since these features cannot be easily changed. However, it is obvious that the physical features as vocal tract dimensions of an unknown speaker cannot be simply measured. Thus, numerical values for physical features or parameters would have to be derived from digital signal processing parameters extracted from the speech signal. Suppose that vocal tracts could be effectively represented by 10 independent physical features, with each feature taking on one of 10 discrete values. In this case, 1010 individuals in the population (i.e., 10 billion) could be distinguished whereas today’s world population amounts to approximately 7 billion individuals. People can reliably identify familiar voices. About 2-3 seconds of speech is sufficient to identify a voice, although performance decreases for unfamiliar voices. One review of human speaker recognition (Lancker et al., 1985) notes that many studies of 8-10 speakers (work colleagues) yield in excess of 97% accuracy if a sentence or more of the test speech is heard. Performance falls to about 54% when duration is shorter than 1 second and/or distorted e.g., severely highpass or lowpass filtered. Performance also falls significantly if training and test utterances are processed through different transmission systems. A study

Automatic speaker recognition determines whether a person is who he/she claims to be without the intervention of another human being, from their voiceprint. In recent years different methods of voice analysis to fulfill this objective have been developed. At the moment, the development of computational auditory models that emulate the physiology and function of the inner ear have added a new tool in the field of speaker recognition, including the Triple Resonance Nonlinear filter. This paper studies the behavior of a bio-inspired model of inner ear applied to the speakers's voice analysis. Their ability on speaker recognition tasks was statistically analyzed. We conclude, given the results obtained, that this system is an excellent tool, which shows a high rate of sensitivity and specificity in speaker recognition.

In the continuous integration, test case prioritization can effectively alleviate the resource-intensive problems associated with frequent integration commits. Test case prioritization in continuous integration is a sequential decision problem from which reinforcement learning is applied and can effectively adapt and learn from a changing environment. However, continuous integration testing brings new problems of sparse rewards to reinforcement learning because of frequent integration with low test failure and this problem can be addressed by increasing the number of rewarded test cases. In this paper, we propose a reinforcement learning reward object selection strategy based on Test Case Synchronization and Diversity (TCSD) that rewards failed test cases and with an additional selection of passed test cases with potential failure ability. The experiments on six real-world industrial data sets show that TCSD improves the learning efficiency and fault detection ability of reinforcement learning 6.35% in average NAPFD compared with the traditional strategies.

Chapter 7 - Reinforcement learning

Voice biometrics are being implemented globally in large scale applications such as remote banking, government e-services, transportation and building security access, autonomous vehicles, and healthcare. They have been integrated in numerous apps, often coupled with face biometrics and artificial intelligence methods. Voice biometrics products and solutions must meet three key re-quirements for the success in their deployment: they must be highly trustable regarding privacy protection; easy to use and always be available. This edited book presents the state of the art in voice biometrics research and technologies including implementation and deployment challenges in terms of interoperability, scalability and performance, and security. The team of editors and chapter authors combine a wealth of expertise from academia and the industry. Topics covered include the fundamentals of voice biometrics; design of countermeasures for replay attack; attacker's perspective for voice biometrics; voice biometrics; speaker de-identification; performance evaluation of voice biometrics solutions; standardization of voice biometrics technology; industry perspectives; joining forces of voice and facial biometrics; and future trends and challenges in voice biometrics. Providing comprehensive coverage of the field of voice biometrics, this authoritative volume will be of great interest to researchers, scientists, engineers, practitioners and advanced students involved in the fields of security, biometrics, forensic sciences, human computer interaction, speech processing, acoustics, multimedia, pattern recognition, and privacy-preserving, digital signal processing and speech technologies. It will also be of interest to researchers and professionals working in law and criminology.

Reinforcement learning (RL) is a computational approach to reward-driven learning in sequential decision problems. It implements the discovery of optimal actions by learning from an agent interacting with an environment rather than from supervised data. We contrast and compare RL with traditional sequential design, focusing on simulation-based Bayesian sequential design (BSD). Recently, there has been an increasing interest in RL techniques for healthcare applications. We introduce two related applications as motivating examples. In both applications, the sequential nature of the decisions is restricted to sequential stopping. Rather than a comprehensive survey, the focus of the discussion is on solutions using standard tools for these two relatively simple sequential stopping problems. Both problems are inspired by adaptive clinical trial design. We use examples to explain the terminology and mathematical background that underlie each framework and map one to the other. The implementations and results illustrate the many similarities between RL and BSD. The results motivate the discussion of the potential strengths and limitations of each approach.

With the increasing number and popularity of smart devices in the past few decades, especially those that execute different types of health-related applications, their security is a growing concern. Biometric authentication can be used for device security, and speaker recognition (SR) is one of its elegant forms. In this study, an SR system is developed for the security of smart devices, which identifies a person from the shortest utterance of Arabic language, achieving the highest SR rate, by using the benchmark Linguistic Data Consortium (LDC) Arabic dataset. This study focuses on the use of the consonants and vowels of the Arabic language. It is observed that certain consonants or a fusion of certain consonants in Arabic language helps to ascertain the speaker’s identity efficiently. The shortest utterance that better presents the speaker’s identity by using such consonants is revealed. The system was analyzed by using different numbers of consonants and achieved the maximum (100%) SR rate using a fusion of only three consonants of the Arabic language. These consonants or their combinations can be used to develop text for any speaker-based authentication system.

The goal of this paper is to review automatic systems for forensic speaker recognition (FSR) based on scientifically approved methods for calculation and interpretation of biometric evidence. The objective of this paper is not to promote one speaker recognition method against another, but is to make available to the biometric research community data-driven methodology combining automatic speaker recognition techniques and a rigorous forensic experimental background. Forensic speaker recognition is the process of determining if a specific individual (suspected speaker) is the source of a questioned speech recording (trace). This paper aims at reviewing forensic automatic speaker recognition (FASR) methods that provide a coherent way of quantifying and presenting recorded speech as biometric evidence, as well as the assessment of its strength (likelihood ratio) in the Bayesian interpretation framework compatible with interpretations in other forensic disciplines. Forensic speaker recognition has proven an effective tool in the fight against crime, yet there is a constant need for more research due to the difficulties involved because of the within-speaker (within-source) variability, between-speakers (between-sources) variability, and differences in recording sessions conditions.KeywordsLikelihood RatioGaussian Mixture ModelVector QuantizationForensic ScienceSpeaker RecognitionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Introduction Reinforcement learning formalizes the concept of learning from interactions.1 Broadly, reinforcement learning focuses on a setting in which an agent (decision maker) sequentially interacts with an environment that is partially unknown to them. At each stage, the agent takes an action and receives a reward. The objective of the agent is to maximize rewards accumulated in the long run. There are many situations in health care where decisions are made sequentially for which reinforcement learning approaches could prove useful for decision making. Throughout this article, we consider treatment prescription as an archetypical example to connect reinforcement learning concepts to a health care setting. In this setting, the care provider, the prescribed treatment, and the patients can be viewed as the agent, the action, and the environment, respectively, as depicted in Figure 1.Figure 1: Sequential treatment of AKI or CKD complications modeled as a reinforcement learning problem.Background In this section, with the objective of making reinforcement learning literature more accessible to a clinical audience, we briefly introduce related fundamental concepts and approaches. We refer the interested reader to Sutton and Barto1 for a comprehensive introduction to reinforcement learning. Markov Decision Processes Markov decision processes (MDPs) are a formalism of the sequential decision-making problem that has been central to the theoretical and practical advancements of reinforcement learning. In each stage of an MDP, the agent observes the state of the environment and takes an action, which, in turn, results in a change of the state. This change of state is assumed to be probabilistic with the next state being determined only by the preceding state, the chosen action, and the transition probability. The agent also receives a reward that is a function of the taken action, the preceding state, and the subsequent state. In an MDP, the objective of the agent is to maximize the return defined as the reward accumulated over a time horizon. In some applications, it is common to consider the horizon to be infinite, in which case the future rewards are discounted by a factor smaller than one. The selection of action by the agent on the basis of the observed state is known as the policy. More formally, a policy is a probabilistic mapping from states to each possible action. Because the policy and the reward are a function of the state, it is critical to estimate the utility of being in a certain state. More specifically, the value function is defined as the expected return starting from a given state under the chosen policy. Under this formalism, the objective of the agent is to find the optimal policy that maximizes the value function for all states. Reinforcement Learning Methods Action-value methods are a class of reinforcement learning methods in which the actions are chosen on the basis of the estimation of their long-term value. A prominent example of an action-value method is Q-learning in which the agent iteratively takes actions with the highest estimated values and updates the action-state value function on the basis of new observations. Policy gradient methods are another class of reinforcement learning methods that seek to optimize the policy directly instead of choosing actions on the basis of their respective estimated value. Such methods could be advantageous in health care applications that entail a large number of possible actions, e.g., when recommending a wide range of drug dosages or treatment options. Clinical Applications Reinforcement learning frameworks and methods are broadly applicable to clinical settings in which decisions are made sequentially. A prominent clinical application of reinforcement learning is for treatment recommendation, which has been studied across a variety of diseases and treatments including radiation and chemotherapy for cancer, brain stimulation for epilepsy, and treatment strategies for sepsis.2–5 In such treatment recommendation settings, a policy is commonly known as a dynamic treatment regime. There are various other clinical applications of reinforcement learning including diagnosis, medical imaging, and decision support tools (see refs. 2–5 and the references therein). Reinforcement Learning in Nephrology Although there have been recent applications of machine learning in nephrology,6,7 to the best of the authors' knowledge, the application of reinforcement learning to nephrology has been primarily limited to optimizing the erythropoietin dosage in hemodialysis patients.8,9 However, there are other settings where reinforcement learning has the potential to improve patient care in nephrology. For example, reinforcement learning methods can be adopted in the treatment of the complications of AKI or CKD (Figure 1). In this problem, the state models the conditions of the patient (e.g., vital signs, laboratory test results including urine and blood tests, and urine output measurements). The action refers to the treatment options (e.g., the dosage of medications such as sodium polystyrene sulfonate, and hemodialysis). The reward models the improvement in patient conditions. Similarly, reinforcement learning can help automate and optimize the dosage of immunosuppressive drugs in kidney transplants. Challenges and Opportunities Despite the success of reinforcement learning in several simplified clinical settings, their large-scale application to patient care faces several open challenges. The complexity of human biology complicates modeling clinical decision making as a reinforcement learning problem. The state space in such settings is often enormous, which could make a purely computational approach infeasible. Moreover, modeling all potential objectives a priori as a reward function may not be feasible. To overcome these challenges and realize the potential of reinforcement learning, clinical insight can play a pivotal role. More specifically, restricting the state space to only include highly relevant clinical variables could greatly reduce the computational complexity. Furthermore, using inverse reinforcement learning,2 relevant reward functions can be learned from retrospective studies assuming the optimality of clinical decisions. Another critical challenge is addressing moral and ethical concerns. It is imperative to ensure that reinforcement learning methods do not cause harm to the patient. To this end, there exists a need for a thorough validation of such methods before their use in patient care. Hence, there is a need to go beyond retrospective studies that have been used for the proof of concept of most existing reinforcement learning methods in health care applications.2,3 The lessons learned from the success of reinforcement learning in other application areas (e.g., self-driving cars) can help navigate the path to realizing its potential in health care. Accessible open-source simulation environments that enable researchers to compare various approaches are essential to the field of reinforcement learning. OpenAI Gym is currently the leading toolkit containing a wide range of simulated environments, e.g., surgical robotics.10 The development of high-quality and reliable simulation environments for nephrology and other health care applications can facilitate the development and validation of reinforcement learning methods beyond limited retrospective studies. The adoption of methods validated in such simulation environments in actual clinical settings will require clinicians' oversight. Similar to how self-driving cars require a human driver to ensure collision avoidance, clinicians' oversight is critical to ensure the safety of the patients, especially in the early stages of the adoption of reinforcement learning methods. The data from clinicians' decisions (e.g., overruling the automated treatment recommendation) can be used to improve the reliability of autonomous systems over time and reduce the burden of clinicians' oversight.

The paper introduces a combination of adaptive neural network systems and statistical method for integrating speech and face image information for person identification. The method allows for the development of models of persons and their on-going adjustment based on new speech and face images. The method is illustrated with a modeling and classification of different persons, when speech and face images are presented in an incremental way. In this model, there are two subnetworks, one for face image and one for speaker recognition. A higher-level layer is applied to make a final decision. In the speaker recognition subnetwork, a text-dependant model is built using evolving connectionist systems (ECOS) [N. Kasabov, 2002]. In the face image recognition subnetwork, composite profile technique is applied for face image feature extraction and zero instruction set computing (ZISC) [ZISC Manual, 2000] technology is used to build the neural network. In the higher-level conceptual subsystem, final recognition decision is made using statistical method. The experiments show that ECOS and ZISC are appropriate techniques for the creation of evolving models for the task of speaker and face recognition individually. It is also shown that the integration of the speech and image information using statistical method improves the person identification rate.