NIST Speech Processing Evaluations: LVCSR, Speaker Recognition, Language Recognition

Relevance factor of maximum a posteriori adaptation for GMM–NAP–SVM in speaker and language recognition

Voice biometrics are being implemented globally in large scale applications such as remote banking, government e-services, transportation and building security access, autonomous vehicles, and healthcare. They have been integrated in numerous apps, often coupled with face biometrics and artificial intelligence methods. Voice biometrics products and solutions must meet three key re-quirements for the success in their deployment: they must be highly trustable regarding privacy protection; easy to use and always be available. This edited book presents the state of the art in voice biometrics research and technologies including implementation and deployment challenges in terms of interoperability, scalability and performance, and security. The team of editors and chapter authors combine a wealth of expertise from academia and the industry. Topics covered include the fundamentals of voice biometrics; design of countermeasures for replay attack; attacker's perspective for voice biometrics; voice biometrics; speaker de-identification; performance evaluation of voice biometrics solutions; standardization of voice biometrics technology; industry perspectives; joining forces of voice and facial biometrics; and future trends and challenges in voice biometrics. Providing comprehensive coverage of the field of voice biometrics, this authoritative volume will be of great interest to researchers, scientists, engineers, practitioners and advanced students involved in the fields of security, biometrics, forensic sciences, human computer interaction, speech processing, acoustics, multimedia, pattern recognition, and privacy-preserving, digital signal processing and speech technologies. It will also be of interest to researchers and professionals working in law and criminology.

This paper presents procedures used for development of a Croatian large vocabulary automatic speech recognition system (LVASR). The proposed acoustic model is based on context-dependent triphone hidden Markov models and Croatian phonetic rules. Different acoustic and language models, developed using a large collection of Croatian speech, are discussed and compared. The paper proposes the best feature vectors and acoustic modeling procedures using which lowest word error rates for Croatian speech are achieved. In addition, Croatian language modeling procedures are evaluated and adopted for speaker independent spontaneous speech recognition. Presented experiments and results show that the proposed approach for automatic speech recognition using context-dependent acoustic modeling based on Croatian phonetic rules and a parameter tying procedure can be used for efficient Croatian large vocabulary speech recognition with word error rates below 5%.

Speaker and language recognition in noisy and degraded channel conditions continue to be a challenging problem mainly due to the mismatch between clean training and noisy test conditions. In the presence of noise, the most reliable portions of the signal are the high energy regions which can be used for robust feature extraction. In this paper, we propose a front end processing scheme based on autoregressive (AR) models that represent the high energy regions with good accuracy followed by a modulation filtering process. The AR model of the spectrogram is derived using two separable time and frequency AR transforms. The first AR model (temporal AR model) of the sub-band Hilbert envelopes is derived using frequency domain linear prediction (FDLP). This is followed by a spectral AR model applied on the FDLP envelopes. The output 2-D AR model represents a low-pass modulation filtered spectrogram of the speech signal. The band-pass modulation filtered spectrograms can further be derived by dividing two AR models with different model orders (cut-off frequencies). The modulation filtered spectrograms are converted to cepstral coefficients and are used for a speaker recognition task in noisy and reverberant conditions. Various speaker recognition experiments are performed with clean and noisy versions of the NIST-2010 speaker recognition evaluation (SRE) database using the state-of-the-art speaker recognition system. In these experiments, the proposed front-end analysis provides substantial improvements (relative improvements of up to 25%) compared to baseline techniques. Furthermore, we also illustrate the generalizability of the proposed methods using language identification (LID) experiments on highly degraded high-frequency (HF) radio channels and speech recognition experiments on noisy data.

Acquiring a foreign language requires a student to master the two elements of language, vocabulary and grammar, and the four language skills, listening, speaking, reading, and writing. Within each skill, these two language elements play a considerably essential role since they are seen as the very core aspects of language learning and that they are regarded inherent in each of these skills. Thus, vocabulary and grammatical aspects cannot be separated from the four language skills in language acquisition as without any one of which each skill is never acquired. Since the implementation of communicative approach in 1984 to the application of the competency-based language teaching in Indonesia, English programs at high schools have been meaning or message based rather than language usage or form oriented. As a result, grammar learning has been significantly kept aside if not totally ignored in class interactions, and that communication success becomes the main target in instructional objectives. This study attempts to map grammatical errors the students make in their speech productions. It employs an interview technique for data collection by recording the conversations of twenty respondents - - junior and senior students - - on a selected issue of a particular topic and transcribing them in the form of written transcripts which are then to be analyzed from a grammatical perspective with the reference of grammar books. Those violating the grammatical norms will be considered as grammatical errors. It has been found out that learners’ grammatical errors are derived from two major causes - - translating concepts of L1 into L2 and their approximative system. This study concludes that the communicative classrooms which utilize the competency-based language teaching bring about a positive impact upon grammar learning. This research is significant as it gives a great contribution to structure and speaking class teachers, students, as well as to the department for policy making.

Speech and speaker recognition technology has made very significant progress in the past 50 years. The progress can be summarized by the following changes: (1) from template matching to corpus-base statistical modeling, e.g., HMM and n-grams, (2) from filter bank/spectral resonance to Cepstral features (Cepstrum + DCepstrum + DDCepstrum), (3) from heuristic time-normalization to DTW/DP matching, (4) from gdistanceh-based to likelihood-based methods, (5) from maximum likelihood to discriminative approach, e.g., MCE/GPD and MMI, (6) from isolated word to continuous speech recognition, (7) from small vocabulary to large vocabulary recognition, (8) from context-independent units to context-dependent units for recognition, (9) from clean speech to noisy/telephone speech recognition, (10) from single speaker to speaker-independent/adaptive recognition, (11) from monologue to dialogue/conversation recognition, (12) from read speech to spontaneous speech recognition, (13) from recognition to understanding, (14) from single-modality (audio signal only) to multi-modal (audio/visual) speech recognition, (15) from hardware recognizer to software recognizer, and (16) from no commercial application to many practical commercial applications. Most of these advances have taken place in both the fields of speech recognition and speaker recognition. The majority of technological changes have been directed toward the purpose of increasing robustness of recognition, including many other additional important techniques not noted above.

Acoustical communication is one of the fundamental prerequisites for the existence of human society. Textual language has become extremely important in modern life, but speech has dimensions of richness that text cannot approximate. From speech alone, fairly accurate guesses can be made as to whether the speaker is male or female, adult or child. In addition, experts can extract from speech information regarding e.g. the speaker’s state of mind. As computer power increased and knowledge about speech signals improved, research of speech processing became aimed at automated systems for many purposes. Speaker recognition is the complement of speech recognition. Both techniques use similar methods of speech signal processing. In automatic speech recognition, the speech processing approach tries to extract linguistic information from the speech signal to the exclusion of personal information. Conversely, speaker recognition is focused on the characteristics unique to the individual, disregarding the current word spoken. The uniqueness of an individual’s voice is a consequence of both the physical features of the person vocal tract and the person mental ability to control the muscles in the vocal tract. An ideal speaker recognition system would use only physical features to characterize speakers, since these features cannot be easily changed. However, it is obvious that the physical features as vocal tract dimensions of an unknown speaker cannot be simply measured. Thus, numerical values for physical features or parameters would have to be derived from digital signal processing parameters extracted from the speech signal. Suppose that vocal tracts could be effectively represented by 10 independent physical features, with each feature taking on one of 10 discrete values. In this case, 1010 individuals in the population (i.e., 10 billion) could be distinguished whereas today’s world population amounts to approximately 7 billion individuals. People can reliably identify familiar voices. About 2-3 seconds of speech is sufficient to identify a voice, although performance decreases for unfamiliar voices. One review of human speaker recognition (Lancker et al., 1985) notes that many studies of 8-10 speakers (work colleagues) yield in excess of 97% accuracy if a sentence or more of the test speech is heard. Performance falls to about 54% when duration is shorter than 1 second and/or distorted e.g., severely highpass or lowpass filtered. Performance also falls significantly if training and test utterances are processed through different transmission systems. A study

Phonetic information is one of the most essential components of a speech signal, playing an important role for many speech processing tasks. However, it is difficult to integrate phonetic information into speaker verification systems since it occurs primarily at the frame level while speaker characteristics typically reside at the segment level. In deep neural network-based speaker verification, existing methods only apply phonetic information to the frame-wise trained speaker embeddings. To improve this weakness, this paper proposes phonetic adaptation and hybrid multi-task learning and further combines these into c-vector and simplified c-vector architectures. Experiments on National Institute of Standards and Technology (NIST) speaker recognition evaluation (SRE) 2010 show that the four proposed speaker embeddings achieve better performance than the baseline. The c-vector system performs the best, providing over 30% and 15% relative improvements in equal error rate (EER) for the core-extended and 10 s–10 s conditions, respectively. On the NIST SRE 2016, 2018, and VoxCeleb datasets, the proposed c-vector approach improves the performance even when there is a language mismatch within the training sets or between the training and evaluation sets. Extensive experimental results demonstrate the effectiveness and robustness of the proposed methods.

Speech, languages, and background voices are overlapped in the multi-speaker environment. The speech separation process is necessary for downstream applications like language and speaker identification and ASR tasks. A combined speech separation and language recognition approach is proposed in this paper. First, the monolingual information is extracted from a multi-speaker multilingual speech scenario, then fed the separated speech to a language identification model. The speech separation model is trained in a single-channel overlapped speech data, and the language identification model is trained in Oriental language corpus. The combined effect of these two models is evaluated in the task of language identification. When the language recognition network is fed with the audio spectrograms separated by the source separation network, the Chinese and Korean language recognition results are significantly improved in terms of accuracy and recall rate than the mixed audio spectrograms.

In global supply chain, the scheduling problems of large products such as ships, airplanes, space shuttles, assembled constructions, and/or automobiles are complicated by nature. New scheduling systems are often developed in order to reduce inherent computational complexity. As a result, a problem can be decomposed into small sub-problems, problems that contain independently small scheduling systems integrating into the initial problem. As one of the authors experienced, DAS (Daewoo Shipbuilding Scheduling System) has adopted a two-layered hierarchical architecture. In the hierarchical architecture, individual scheduling systems composed of a high-level dock scheduler, DAS-ERECT and low-level assembly plant schedulers, DAS-PBS, DAS-3DS, DAS-NPS, and DAS-A7 try to search the best schedules under their own constraints. Moreover, the steep growth of communication technology and logistics enables it to introduce distributed multi-nation production plants by which different parts are produced by designated plants. Therefore vertical and lateral coordination among decomposed scheduling systems is necessary. No standard coordination mechanism of multiple scheduling systems exists, even though there are various scheduling systems existing in the area of scheduling research. Previous research regarding the coordination mechanism has mainly focused on external conversation without capacity model. Prior research has heavily focuses on agent-based coordination in the area of agent research. Yet, no scheduling domain has been developed. Previous research regarding the agent-based scheduling has paid its ample attention to internal coordination of scheduling process, a process that has not been efficient. In this study, we suggest a general framework for agent-based coordination of multiple scheduling systems in global supply chain. The purpose of this study was to design a standard coordination mechanism. To do so, we first define an individual scheduling agent responsible for their own plants and a meta-level coordination agent involved with each individual scheduling agent. We then suggest variables and values describing the individual scheduling agent and meta-level coordination agent. These variables and values are represented by Backus-Naur Form. Second, we suggest scheduling agent communication protocols for each scheduling agent topology classified into the system architectures, existence or nonexistence of coordinator, and directions of coordination. If there was a coordinating agent, an individual scheduling agent could communicate with another individual agent indirectly through the coordinator. On the other hand, if there was not any coordinating agent existing, an individual scheduling agent should communicate with another individual agent directly. To apply agent communication language specifically to the scheduling coordination domain, we had to additionally define an inner language, a language that suitably expresses scheduling coordination. A scheduling agent communication language is devised for the communication among agents independent of domain. We adopt three message layers which are ACL layer, scheduling coordination layer, and industry-specific layer. The ACL layer is a domain independent outer language layer. The scheduling coordination layer has terms necessary for scheduling coordination. The industry-specific layer expresses the industry specification. Third, in order to improve the efficiency of communication among scheduling agents and avoid possible infinite loops, we suggest a look-ahead load balancing model which supports to monitor participating agents and to analyze the status of the agents. To build the look-ahead load balancing model, the status of participating agents should be monitored. Most of all, the amount of sharing information should be considered. If complete information is collected, updating and maintenance cost of sharing information will be increasing although the frequency of communication will be decreasing. Therefore the level of detail and updating period of sharing information should be decided contingently. By means of this standard coordination mechanism, we can easily model coordination processes of multiple scheduling systems into supply chain. Finally, we apply this mechanism to shipbuilding domain and develop a prototype system which consists of a dock-scheduling agent, four assembly- plant-scheduling agents, and a meta-level coordination agent. A series of experiments using the real world data are used to empirically examine this mechanism. The results of this study show that the effect of agent-based platform on coordinated scheduling is evident in terms of the number of tardy jobs, tardiness, and makespan.

Introduction: In order to assess how well conventional and hybrid pitch detection techniques perform in speech processing applications, a comparative analysis of the two types of methods is conducted. Method: A proposed hybrid approach, Proposed PEF+CEP, is examined alongside five traditional algorithms, namely Normalized Correlation Function (NCF), Pitch Estimation Filter (PEF), Log- Harmonic Summation (LHS), Summation of Residual Harmonics (SRH) and Cepstrum Pitch Determination (CEP). The effectiveness is evaluated using performance metrics like accuracy, specificity, sensitivity, and Gross Pitch Error (GPE). Results and Discussion: Our findings show that the accuracy and specificity of the traditional methods are impressive; the accuracy and sensitivity of the suggested hybrid method surpass their performance, with an astounding 98.8% accuracy and 99.2% sensitivity. Conclusion: Furthermore, the Proposed PEF+CEP method is a promising solution for accurate and dependable pitch detection in speech processing applications because it strikes a strong balance between computational efficiency and robustness. These results open up new avenues for research in the field of speech processing and demonstrate the potential of hybrid approaches.

For communication, speech is one of the natural forms. A person’s voice contains various parameters that convey information such as emotions, gender, attitude, health, and identity. Determination of these parameters will help to further develop the technology into a reliable and consistent means of identification using speaker recognition. Speaker recognition technologies have wide application areas especially in authentication; surveillance and forensic speaker recognition. In addition, speaker recognition refers to the automated method of identifying or confirming the identity of an individual based on his/her voice. Speech recognition strips out the personal differences to detect the words. Speaker recognition typically disregards the language and meaning to detect the physical person behind the speech. Speech recognition is language-dependent, while Speaker recognition is independent of language. In essence, voice biometrics provides speaker recognition rather than speech recognition. The most accepted form of identification for a human is his/her speech signal. Principally the speaker recognition is the computing task of validating a user’s claimed identity 108using characteristics extracted from their voice. The speaker recognition process based on a speech signal is treated as one of the most exciting technologies of human recognition. For Speaker identification activities we mainly emphasize the physical features of signal. Speakers could be categorized as speaker identification and speaker verification. In speaker identification, the obtained features are compared with all the speaker’s features which are stored in a voice model database and in speaker verification the obtained features are only compared with the stored features of the speaker he/she claimed to be. In this chapter the general principles of speaker recognition, methodology, and applications are discussed.

<b>Purpose:</b> The aim of the study was to assess whether sequential cochlear implantation (CI) with a prolonged interimplant interval (<i>M </i>= 15.2 years) between the first and second CIs benefited speech recognition and health-related quality of life.<b>Method: </b>This prospective study included 14 prelingually deafened participants who received their second CI after a prolonged interimplant interval (<i>M</i> = 15.2 years). Additionally, speech recognition ability over a 12-month period of bilateral implant use was investigated. The results of the speech recognition test in both quiet and noisy conditions were statistically analyzed for each CI alone and both CIs together. Nijmegen Cochlear Implant Questionnaire scores were also collected at activation and at 12 months after activation.<b>Results:</b> Improvements in speech recognition ability were observed following the use of the first implant alone and with the use of both implants together; however, progress was much slower with the use of the second implant alone, following its introduction. Furthermore, a significant difference in the trajectory of speech recognition ability was observed between the first and the second implanted ear. According to Nijmegen Cochlear Implant Questionnaire scores, all participants benefitted from bilateral CI after 12 months.<b>Conclusions: </b>Prolonged interimplant intervals resulted in asymmetrical speech recognition abilities. A significant improvement in the speech recognition scores was observed with the first implanted ear, and much slower progress was observed with the second implanted ear. However, the “poorer” second implanted ear could provide a considerable beneficial effect on the improved speech recognition and health-related quality of life with the bilateral CI.<br><b>Supplemental Material S1. </b>Arcsine transform; comparison of recognition scores with three modalities over time; predictors of diverge time point.<br>Zhang, Z., Wei, C., Zhang, Y., Zeng, Z., Cao, K., &amp; Liu, Y. (2020). Sequential bilateral cochlear implantation with prolonged time intervals. <i>Journal of Speech, Language, and Hearing Research</i>. https://doi.org/10.1044/2020_JSLHR-20-00140

The development of the i-vector framework for generating low dimensional representations of speech utterances has led to considerable improvements in speaker recognition performance. Although these gains have been achieved in periodic National Institute of Standards and Technology (NIST) evaluations, the problem of domain mismatch, where the system development data and the application data are collected from different sources, remains a challenging one. The impact of domain mismatch was a focus of the Johns Hopkins University (JHU) 2013 speaker recognition workshop, where a domain adaptation challenge (DAC13) corpus was created to address this problem. This paper proposes an approach to domain mismatch compensation for applications where in-domain development data is assumed to be unavailable. The method is based on a generalization of data whitening used in association with i-vector length normalization and utilizes a library of whitening transforms trained at system development time using strictly out-of-domain data. The approach is evaluated on the 2013 domain adaptation challenge task and is shown to compare favorably to in-domain conventional whitening and to nuisance attribute projection (NAP) inter-dataset variability compensation.

Automatic Speaker Recognition systems show interesting properties, such as speed of processing or repeatability of results, in contrast to speaker recognition by humans. But they will be usable just if they are reliable. Testability, or the ability to extensively evaluate the goodness of the speaker detector decisions, becomes then critical. In the last 20 years, the US National Institute of Standards and Technology (NIST) has organized, providing the proper speech data and evaluation protocols, a series of text-independent Speaker Recognition Evaluations (SRE). Those evaluations have become not just a periodical benchmark test, but also a meeting point of a collaborative community of scientists that have been deeply involved in the cycle of evaluations, allowing tremendous progress in a specially complex task where the speaker information is spread across different information levels (acoustic, prosodic, linguistic…) and is strongly affected by speaker intrinsic and extrinsic variability factors. In this paper, we outline how the evaluations progressively challenged the technology including new speaking conditions and sources of variability, and how the scientific community gave answers to those demands. Finally, NIST SREs will be shown to be not free of inconveniences, and future challenges to speaker recognition assessment will also be discussed.