Unsupervised Polyglot Text-to-speech

Target language extraction (TLE) is a novel task in the field of selective auditory attention, which seeks to extract all speech signals that are spoken in a target language from other sources in a multilingual cocktail party. In our prior studies, a TLE model was trained to extract a predefined, single target language, referred to as Single-TLE. In this paper, we extend the Single-TLE framework to Multi-TLE. Multi-TLE models can also extract all speech signals of one specific target language, but they are optimized on a set of multiple target languages during training. As such, they learn the characteristics of several target languages and can replace multiple Single-TLE models without retraining. We perform experiments on the GlobalPhoneMCP database and incorporate a dynamic language mixing scheme for training. The Multi-TLE model does not only outperform Single-TLE models, but when given a language ID as additional input, it is also able to extract the speech of a specific target language from a mixture which contains multiple learned target languages.

We study a new problem of cross-lingual transfer learning for event coreference resolution (ECR) where models trained on data from a source language are adapted for evaluations in different target languages. We introduce the first baseline model for this task based on XLM-RoBERTa, a state-of-the-art multilingual pre-trained language model. We also explore language adversarial neural networks (LANN) that present language discriminators to distinguish texts from the source and target languages to improve the language generalization for ECR. In addition, we introduce two novel mechanisms to further enhance the general representation learning of LANN, featuring: (i) multi-view alignment to penalize cross coreference-label alignment of examples in the source and target languages, and (ii) optimal transport to select close examples in the source and target languages to provide better training signals for the language discriminators. Finally, we perform extensive experiments for cross-lingual ECR from English to Spanish and Chinese to demonstrate the effectiveness of the proposed methods.

The present study proposes a source speech register analysis model that incorporates textual and situational components and parameters of register equivalence in the target language to identify the relations between the source language register and student interpreter’s performance, textual register mismatch between the source and target languages, and the relevance between register equivalence parameters in the target language and interpreting quality. The findings are as follows: first, source language register does have influence on interpreter’s performance, and relations exist between the two. In general, higher source language register brings better interpreting quality. Three assessment criteria and the overall quality differ across different levels of registers in the source language. Second, register mismatch between the source and target languages happens since interpreters are inclined to lower the register of interpreting output from the high-register source speech and increase the register level in the target language in the case of the low-register speech. Third, intonation and voice have the highest correlations with the interpreters’ performance. Fluency ranks the second, and lexical-semantic choices the third.

The subspace Gaussian mixture model (SGMM) has been exploited for cross-lingual speech recognition. The general motivation is that the subspace parameters can be estimated on multiple source languages and then transferred to the target language. In this work, we investigate an extension to SGMM, referred to as subspace mixture model (SMM), in which subspace parameters on the target language are casted as a linear mixture of the subspaces derived from source languages. This approach reduces the number of SGMM model parameters, while retaining the flexibility of subspace learning on the target language. Experiments show that the proposed SMM method outperforms SGMM significantly when the target language has limited training data.

Speech-to-speech translation (S2ST) is the process by which a spoken utterance in one language is used to produce a spoken output in another language. The conventional approach to has focused on processing linguistic information only by directly translating the spoken utterance from the source language to the target language without taking into account paralinguistic and non-linguistic information such as the emotional states at play in the source language. In this work, we explore how to deal with Para-and non-linguistic information among multiple languages, with a particular focus on speakers' emotional states, in scenarios called affective S2ST. In our efforts to construct an effective system, we discuss (1) how to describe emotions in speech and how to model the perception/production of emotions and (2) the commonality and differences among multiple languages in the proposed model. We then use these discussions as context for (3) an examination of our affective S2ST system in operation.

The study focuses on teaching the translation of audiovisual memes: comic image/video and text combinations that tend to go viral on the Internet. The key features of a meme are a comic effect resulting from surprise, disrupted expectations, the clash between the text and the video; “viral” expansion across the Internet and other social media. Mastering the skills required to translate such audiovisual content is essential since memes are highly popular on the Internet with all age groups, there are numerous allusions references to the popular memes in the mass media, fiction, movies, and TV series. Teaching audiovisual meme translation shall consist of analyzing the meme properties, building meme comprehension and metacognitive analysis skills, and mastering specific approaches and techniques for meme representation in the source and target languages. An emphasis should be placed on developing the ability to adequately reconstruct meme ecosystems in the source language and find their analogs in the target language as well as the skills required to identify archetypical narratives in the source language and build them in the target language. It is shown that memes have some similarities to set expressions and proverbs. The meme translation curriculum shall build the ability to find and translate set expressions and proverbs in audiovisual works. It is also shown that translation teachers should have specialized training in teaching the translation of memes as a special type of audiovisual content. It can be implemented in the format of short seminars/webinars, or within the teacher further education framework.

This paper discusses English-poetry translation, Heusca by John Conford into Indonesian translated by Chairil Anwar. The purpose of this study is to determine how the translator considers aesthetic and expressive values from the source language and translate them into target language. Aesthetic values are the means of delivery thought through the use of poetic language such as diction or word choice, metaphor, imageri, and others figurative language. Meanwhile, the expressive values are the means of delivery thoughts through the author's mind or emotions such as structure, rhyme, and pronunciation. The method used in this study is a qualitative descriptive analysis of the meaning. The results of analysis show that in terms of meaning, aesthetic values, and structures, Heusca is the result of a semantic translation and faithful to source language. Translator managed to retain the meaning of source language. In other word, the translation can meet the requirements of fidelity. In terms of expressive values in particular aspects of meter and rhyme are not so faithful to source language because the source language and the target language have some differences. Basically, this is acceptable that the Indonesian language as target language has a very big difference in terms of pronunciation with English as the source language. As a result, meter and rhyme of target language may not be forced into the source language without damaging the naturalness of meaning for target language readers. It can be said that the translation can meet the transparency criteria because source language appeared in poetry comes naturally to native speakers of English.

We propose a new method to resolve ambiguity in translation and meaning interpretation using linguistic statistics extracted from dual corpora of source and target languages in addition to the logical restrictions described on dictionary and grammar rules for ambiguity resolution. It provides reasonable criteria for determining a suitable equivalent translation or meaning by making the dependency relation in the source language be reflected in the translated text. The method can be tractable because the required statistics can be computed semi-automatically in advance from a source language corpus and a target language corpus, while an ordinal corpus-based translation method needs a large volume of bilingual corpus of strict pairs of a sentence and its translation. Moreover, it also provides the means to compute the linguistic statistics on the pairs of meaning expressions.

http://dx.doi.org/10.5007/2175-7968.2015v35nesp2p289Este texto centra-se nas questões do trabalho do Intérprete de Língua de Sinais na condição de segundo enunciador da produção discursiva. Esses profissionais, no momento da interpretação, buscam recriar [atualizar] os conceitos emanados da língua fonte para a língua alvo, instaurando, assim, uma segunda instância enunciativa. O quadro teórico mobilizado situa-se na perspectiva da Linguística Enunciativa (BENVENISTE, 1989; FLORES, 2008, 2009a, 2009b, 2010) e no estudo dos modais em línguas orais e na LIBRAS. Tem-se, por objetivos, buscar as marcas deixadas pelo segundo sujeito enunciador do texto, neste caso, o intérprete, quanto ao uso de modais especificamente e analisar as escolhas desse segundo sujeito enunciador por utilizar um ou outro tipo de modal (no texto alvo), que esteja explícito ou implícito na primeira situação de enunciação (no texto fonte). Para análise, a fala de uma pessoa surda, com tempo total de 40' (quarenta minutos), que estava sendo interpretada para o Português oral, foi transcrita. Após a transcrição, 6 (seis) excertos da fala total, com ocorrências de modais na língua fonte ou alvo, foram analisados. Em função das análises empreendidas, pontua-se: 1. a tendência de que a modalidade deôntica seja quase sempre interpretada como na língua fonte; 2. as escolhas interpretativas são sempre guiadas por pistas enunciativas deixadas na fala do primeiro enunciador, para as quais o intérprete deve revelar domínio de aspectos pertinentes tanto à língua fonte quanto à língua alvo; 3. na ausência de marcas modais explícitas, os intérpretes são sensíveis às noções modais extravasadas por toda a superfície textual; 4. a omissão temporal na interpretação de modais epistêmicos de grande valor de aderência podendo funcionar como uma estratégia interpretativa e não como um falseamento da informação original; 5. a noção de “tempo de checagem”, sendo proposta para recobrir o tempo (não cronológico) no qual o intérprete, após receber a informação da língua fonte, realizasua interpretação.

The ongoing debate on code switching (CS) in multilingual societies and Foreign Language (FL) classes is still relevant and worth considering. This linguistic phenomenon simply refers to altering from one language to another during speech and writing (Richards and Schmidt 2013). While some authorities propose that interactions between learners and instructors in the FL class be carried out only in the target language, other opinions favour a careful blend of both target and source languages. Ghana, a multilingual country in West Africa, adopted English as its official language during the colonial era. French was later introduced to promote regional integration among Ghana and her neighbours, especially since Ghana is greatly surrounded by francophone countries within the subregion (Gandolfi et al. 2021). French has since then been the second most widely taught FL in the country, with a presence in almost all stages of the Ghanaian academic trajectory. This multilingual position of Ghana makes it common to find elements of CS present in FL classrooms, where both instructors and learners occasionally resort to English for instructional conversations (Chachu 2016). This study focuses on university students’ perceptions of CS between English as the source language and French as a foreign language (FFL) during the instruction of the latter. Some frameworks for teaching and learning FFL in Ghana prefer that the language of instruction and interaction be limited to the target language only (Soku and Azanku 2015). The Common European Framework of References for Languages (CEFR) however, suggests the possible use of CS for purposes of instruction when there are favourable conditions (Council of Europe 2001). To identify the roles of CS and investigate FL students’ opinions on this, our data was gathered from classroom observations and questionnaires that had both Likert scale and open-ended questions. This was at the end of their first semester of studying university-level FFL. It was observed that CS plays a significant role in the understanding of linguistic concepts and greatly improves students’ engagement and participation in the FL class. Of the 36 respondents, 34 felt their academic needs were met when the language of instruction and interaction alternated between French and English. Twenty-nine students reported losing focus easily in the FFL class in instances where the medium of interaction was limited to French alone, while 26 reported feeling nervous whenever they were asked to only communicate in French. For metalinguistic aspects of the language, 33 students, constituting 91.7% were of the view that a careful blend of English and French could go a long way to aid their understanding and foster the learning of grammatical concepts. Generally, students attested to better retention of vocabulary in the target language when there was a good blend of both languages. It is recommended therefore that language instructors resort periodically to CS to ensure understanding and engagement from students. CS also helps learners have the opportunity to intervene with their limited linguistic structures when they are faced with linguistic challenges in the FL classroom. Click here to read the corresponding ISSOTL blog post.

Automatic speech recognition depends on large amounts of transcribed speech recordings in order to estimate the parameters of the acoustic model. Recording such large speech corpora is time-consuming and expensive; as a result, sufficient quantities of data exist only for a handful of languages—there are many more languages for which little or no data exist. Given that there are acoustic similarities between speech in different languages, it may be fruitful to use data from a well-resourced source language to estimate the acoustic models for a recognizer in a poorly-resourced target language. Previous approaches to this task have often involved making assumptions about shared phonetic inventories between the languages. Unfortunately pairs of languages do not generally share a common phonetic inventory. We propose an indirect way of transferring information from a source language acoustic model to a target language acoustic model without having to make any assumptions about the phonetic inventory overlap. To do this, we employ tandem features, in which class-posteriors from a separate classifier are decorrelated and appended to conventional acoustic features. Tandem features have the advantage that the language of the speech data used to train the classifier need not be the same as the target language to be recognized. This is because the class-posteriors are not used directly, so do not have to be over any particular set of classes. We demonstrate the use of tandem features in cross-lingual settings, including training on one or several source languages. We also examine factors which may predict a priori how much relative improvement will be brought about by using such tandem features, for a given source and target pair. In addition to conventional phoneme class-posteriors, we also investigate whether articulatory features (AFs)—a multi-stream, discrete, multi-valued labeling of speech—can be used instead. This is motivated by an assumption that AFs are less language-specific than a phoneme set.

Cross-lingual transfer learning: A PARAFAC2 approach

Artificial intelligence is getting shrewd in real time with massive amounts of computational power and high in demand in last quarter more than billion dollars. The Intensification of Neural Networks in Machine learning playing major revolution in Machine Translation (MT). Looking same aspect, we have prepared a paper on Machine Translation for Hindi using Neural Machine Translation techniques. The Basis of this paper is to translate the source language into target language with the help of sentence structure for source language (English sentence) and corresponding reordering rules for target language (Hindi sentence) using a deep neural network (DNN).When the source input that is English sentences are exactly matched with the target language database then the suitable translation for related input is directly fetched from the database. The fuzzy matching between the input sentence and the sample sentence in the database is done with cosine similarity between the sentence structure of input sentence and the sample sentence and then the reordering rule of the most alike sentence to the input sentence is used for translation.Extracting the structure of the source language sentence and extracting the reordering rule for target sentence is the core part of the translator. Reordering of words in spite of that remains one of the complex problems.It is exciting to implement Deep Neural Network and fuzzy logic to build intelligent reordering rules machine learning system.

In recent decades, statistical approaches have significantly advanced the development of machine translation systems. However, the applicability of these methods directly depends on the availability of very large quantities of parallel data. Recent works have demonstrated that a comparable corpus can compensate for the shortage of parallel corpora. In this paper, we propose an alternative to comparable corpora containing text documents as resources for extracting parallel data: a multimodal comparable corpus with audio documents in source language and text document in target language, built fromEuronewsandTEDweb sites. The audio is transcribed by an automatic speech recognition system, and translated with a baseline statistical machine translation system. We then use information retrieval in a large text corpus in the target language in order to extract parallel sentences/phrases. We evaluate the quality of the extracted data on an English to French translation task and show significant improvements over a state-of-the-art baseline.

We propose a simple modification to existing neural machine translation (NMT) models that enables using a single universal model to translate between multiple languages while allowing for language specific parameterization, and that can also be used for domain adaptation. Our approach requires no changes to the model architecture of a standard NMT system, but instead introduces a new component, the contextual parameter generator (CPG), that generates the parameters of the system (e.g., weights in a neural network). This parameter generator accepts source and target language embeddings as input, and generates the parameters for the encoder and the decoder, respectively. The rest of the model remains unchanged and is shared across all languages. We show how this simple modification enables the system to use monolingual data for training and also perform zero-shot translation. We further show it is able to surpass state-of-theart performance for both the IWSLT-15 and IWSLT-17 datasets and that the learned language embeddings are able to uncover interesting relationships between languages.