Efficient Language Model Construction for Spoken Dialog Systems by Inducting Language Resources of Different Languages

Not all languages and domains of handwriting have large labeled datasets available for training handwriting recognition (HWR) models. One way to address this problem is to leverage high resource languages to help train models for low resource languages. In this work, we adapt HWR models trained on a source language to a target language that uses the same writing script. We do so using only labeled data in the source language, unlabeled data in the target language, and a language model in the target language. The language model is used to produce target transcriptions to allow regular example based training. Using this approach we demonstrate improved transferability among French, English, and Spanish languages using both historical and modern handwriting datasets.

In contrast to traditional rule-based approaches to building spoken dialogue systems, recent research has shown that it is possible to implement all of the required functionality using statistical models trained using a combination of supervised learning and reinforcement learning. This approach to spoken dialogue is based on the mathematics of partially observable Markov decision processes (POMDPs) in which user inputs are treated as observations of some underlying belief state, and system responses are determined by a policy which maps belief states into actions. Virtually all current spoken dialogue systems are designed to operate in either a specific carefully defined domain such as restaurant information and appointment booking, or they have very limited conversational ability such as in Siri and Google Now. However, if voice is to become a significant input modality for accessing web-based information and services, then techniques will be needed to enable conversational spoken dialogue systems to operate within open domains. This talk will discuss methods by which current statistical approaches to spoken dialogue can be extended to cover much wider domains. It will be argued that unlike many other areas of machine learning, spoken dialogue systems always have a user on-hand to provide supervision. Hence spoken dialogue systems provide a unique opportunity to automatically adapt on large quantities of speech data without the need for costly annotation.

We study a new problem of cross-lingual transfer learning for event coreference resolution (ECR) where models trained on data from a source language are adapted for evaluations in different target languages. We introduce the first baseline model for this task based on XLM-RoBERTa, a state-of-the-art multilingual pre-trained language model. We also explore language adversarial neural networks (LANN) that present language discriminators to distinguish texts from the source and target languages to improve the language generalization for ECR. In addition, we introduce two novel mechanisms to further enhance the general representation learning of LANN, featuring: (i) multi-view alignment to penalize cross coreference-label alignment of examples in the source and target languages, and (ii) optimal transport to select close examples in the source and target languages to provide better training signals for the language discriminators. Finally, we perform extensive experiments for cross-lingual ECR from English to Spanish and Chinese to demonstrate the effectiveness of the proposed methods.

We propose an architecture for integrating discourse processing and speech recognition (SR) in spoken dialogue systems. It was first developed for computer-mediated bilingual dialogue in voiceto-voice machine translation applications and we apply it here to a distributed battlefield simulation system used for military training. According to this architecture discourse functions previously distributed through the interface code are collected into a centralized discourse capability. The Dialogue Manager (DM) acts as a third-party mediator overseeing the translation of input and output utterances between English and the command language of the backend system. The DM calls the Discourse Processor (DP) to update the context representation each time an utterance is issued or when a salient non-linguistic event occurs in the simulation. The DM is responsible for managing the interaction among components of the interface system and the user. For task-based human-computer dialogue systems it consults three sources of nonlinguistic context constraint in addition to the linguistic Discourse State: (1) a User Model, (2) a static Domain Model containing rules for engaging the backend system, with a grammar for the language of well-formed, executable commands, and (3) a dynamic Backend Model (BEM) that maintains updated status for salient aspects of the non-linguistic context. In this paper we describe its four-step recovery algorithm invoked by DM whenever an item is unclear in the current context, or when an interpretation error is, and show how parameter settings on the algorithm can modify the overall behavior of the system from Tutor to Trainer. This is offered to illustrate how limited (inexpensive) dialogue processing functionality, judiciously selected, and designed in conjunction with expectations for human dialogue behavior can compensate for inevitable limitations in SR, NL processor, the backend software application, or even in the user’s understanding of the task or the software system. 1. SPOKEN DIALOGUE SYSTEMS 1.1 Integrating Discourse and SR Waibel et al., (1989) and De Mori et al., (1988) extend stochastic language modeling techniques to the discourse level to improve spoken dialogue systems. The complexity of discourse state descriptions leads to a sparse data problem during training, and idiosyncratic human behavior at run time can defeat even the best probabilistic dialogue model. Symbolic approaches to spoken discourse data identify discourse constraints on language model selection at run time. Our work collects discourse-level processing into a centralized discourse capability as part of a modular user interface dialogue architecture. Its use in a spoken dialogue interface to a distributed battlefield simulation system used for military training is diagrammed in Figure 1.

The present study proposes a source speech register analysis model that incorporates textual and situational components and parameters of register equivalence in the target language to identify the relations between the source language register and student interpreter’s performance, textual register mismatch between the source and target languages, and the relevance between register equivalence parameters in the target language and interpreting quality. The findings are as follows: first, source language register does have influence on interpreter’s performance, and relations exist between the two. In general, higher source language register brings better interpreting quality. Three assessment criteria and the overall quality differ across different levels of registers in the source language. Second, register mismatch between the source and target languages happens since interpreters are inclined to lower the register of interpreting output from the high-register source speech and increase the register level in the target language in the case of the low-register speech. Third, intonation and voice have the highest correlations with the interpreters’ performance. Fluency ranks the second, and lexical-semantic choices the third.

The study focuses on teaching the translation of audiovisual memes: comic image/video and text combinations that tend to go viral on the Internet. The key features of a meme are a comic effect resulting from surprise, disrupted expectations, the clash between the text and the video; “viral” expansion across the Internet and other social media. Mastering the skills required to translate such audiovisual content is essential since memes are highly popular on the Internet with all age groups, there are numerous allusions references to the popular memes in the mass media, fiction, movies, and TV series. Teaching audiovisual meme translation shall consist of analyzing the meme properties, building meme comprehension and metacognitive analysis skills, and mastering specific approaches and techniques for meme representation in the source and target languages. An emphasis should be placed on developing the ability to adequately reconstruct meme ecosystems in the source language and find their analogs in the target language as well as the skills required to identify archetypical narratives in the source language and build them in the target language. It is shown that memes have some similarities to set expressions and proverbs. The meme translation curriculum shall build the ability to find and translate set expressions and proverbs in audiovisual works. It is also shown that translation teachers should have specialized training in teaching the translation of memes as a special type of audiovisual content. It can be implemented in the format of short seminars/webinars, or within the teacher further education framework.

Incorporating discourse features into confidence scoring of intention recognition results in spoken dialogue systems

This paper discusses English-poetry translation, Heusca by John Conford into Indonesian translated by Chairil Anwar. The purpose of this study is to determine how the translator considers aesthetic and expressive values from the source language and translate them into target language. Aesthetic values are the means of delivery thought through the use of poetic language such as diction or word choice, metaphor, imageri, and others figurative language. Meanwhile, the expressive values are the means of delivery thoughts through the author's mind or emotions such as structure, rhyme, and pronunciation. The method used in this study is a qualitative descriptive analysis of the meaning. The results of analysis show that in terms of meaning, aesthetic values, and structures, Heusca is the result of a semantic translation and faithful to source language. Translator managed to retain the meaning of source language. In other word, the translation can meet the requirements of fidelity. In terms of expressive values in particular aspects of meter and rhyme are not so faithful to source language because the source language and the target language have some differences. Basically, this is acceptable that the Indonesian language as target language has a very big difference in terms of pronunciation with English as the source language. As a result, meter and rhyme of target language may not be forced into the source language without damaging the naturalness of meaning for target language readers. It can be said that the translation can meet the transparency criteria because source language appeared in poetry comes naturally to native speakers of English.

We consider the pronunciation assessment of vowels of Indian English uttered by speakers with Gujarati L1 using confidence measures obtained by automatic speech recognition. The goodness-of-pronunciation measure as captured by the acoustic likelihood scores can be effective only when the acoustic models used are appropriate for the task i.e. detecting errors in the target language (Indian English) typical of speakers of Gujarati (the source language). Thus the speech data used for acoustic model training is expected to have a prominent influence on system performance. In the absence of labeled speech databases of either the source or target language, we investigate specific combinations of acoustic models trained on available databases of American English and Hindi. It is observed that Indian English speech is better represented by Hindi speech models for vowels common to the two languages rather than by American English models. Further, adaptation with a limited amount of Indian English speech improves the system performance.

We propose a new method to resolve ambiguity in translation and meaning interpretation using linguistic statistics extracted from dual corpora of source and target languages in addition to the logical restrictions described on dictionary and grammar rules for ambiguity resolution. It provides reasonable criteria for determining a suitable equivalent translation or meaning by making the dependency relation in the source language be reflected in the translated text. The method can be tractable because the required statistics can be computed semi-automatically in advance from a source language corpus and a target language corpus, while an ordinal corpus-based translation method needs a large volume of bilingual corpus of strict pairs of a sentence and its translation. Moreover, it also provides the means to compute the linguistic statistics on the pairs of meaning expressions.

http://dx.doi.org/10.5007/2175-7968.2015v35nesp2p289Este texto centra-se nas questões do trabalho do Intérprete de Língua de Sinais na condição de segundo enunciador da produção discursiva. Esses profissionais, no momento da interpretação, buscam recriar [atualizar] os conceitos emanados da língua fonte para a língua alvo, instaurando, assim, uma segunda instância enunciativa. O quadro teórico mobilizado situa-se na perspectiva da Linguística Enunciativa (BENVENISTE, 1989; FLORES, 2008, 2009a, 2009b, 2010) e no estudo dos modais em línguas orais e na LIBRAS. Tem-se, por objetivos, buscar as marcas deixadas pelo segundo sujeito enunciador do texto, neste caso, o intérprete, quanto ao uso de modais especificamente e analisar as escolhas desse segundo sujeito enunciador por utilizar um ou outro tipo de modal (no texto alvo), que esteja explícito ou implícito na primeira situação de enunciação (no texto fonte). Para análise, a fala de uma pessoa surda, com tempo total de 40' (quarenta minutos), que estava sendo interpretada para o Português oral, foi transcrita. Após a transcrição, 6 (seis) excertos da fala total, com ocorrências de modais na língua fonte ou alvo, foram analisados. Em função das análises empreendidas, pontua-se: 1. a tendência de que a modalidade deôntica seja quase sempre interpretada como na língua fonte; 2. as escolhas interpretativas são sempre guiadas por pistas enunciativas deixadas na fala do primeiro enunciador, para as quais o intérprete deve revelar domínio de aspectos pertinentes tanto à língua fonte quanto à língua alvo; 3. na ausência de marcas modais explícitas, os intérpretes são sensíveis às noções modais extravasadas por toda a superfície textual; 4. a omissão temporal na interpretação de modais epistêmicos de grande valor de aderência podendo funcionar como uma estratégia interpretativa e não como um falseamento da informação original; 5. a noção de “tempo de checagem”, sendo proposta para recobrir o tempo (não cronológico) no qual o intérprete, após receber a informação da língua fonte, realizasua interpretação.

• Speech recognition and language understanding performance can be improved through an OOV expectand-learn procedure. • A limited domain vocabulary can be utilized to effectively acquire OOVs by the word relatedness theory through web knowledge bases. • With data-driven semantic relatedness, both the global and local learning procedures are able to successfully harvest more than 50% of OOVs, leading to better recognition and understanding performance. • This work demonstrates that o OOV learning may benefit dialog system o the proposed expect-and-learn strategy outperforms the traditional detect-and-learn in both higher effectiveness and no human involvement. 1. Linguistically semantic relatedness o Defined by linguistics, e.g., WordNet (WN), Paraphrase Database (PPDB) (Ganitkevitch et al., 2013) 2. Data-driven semantic relatedness o Distributional semantics, e.g., continuous bag-ofword embeddings (CBOW) (Mikolov et al., 2013)  Detect-and-Learn (Qin et al., 2011; 2012): o Discover OOV words during the conversation o Example: S: “I heard something like SELF, can you repeat it?” U: “It’s SELFIE.” o Drawbacks • Limited number of new words • Required human efforts to correct spellings and pronunciations  Expect-and-Learn (proposed): o Use semantic relatedness to automatically enrich the vocabulary and language model beforehand

The ongoing debate on code switching (CS) in multilingual societies and Foreign Language (FL) classes is still relevant and worth considering. This linguistic phenomenon simply refers to altering from one language to another during speech and writing (Richards and Schmidt 2013). While some authorities propose that interactions between learners and instructors in the FL class be carried out only in the target language, other opinions favour a careful blend of both target and source languages. Ghana, a multilingual country in West Africa, adopted English as its official language during the colonial era. French was later introduced to promote regional integration among Ghana and her neighbours, especially since Ghana is greatly surrounded by francophone countries within the subregion (Gandolfi et al. 2021). French has since then been the second most widely taught FL in the country, with a presence in almost all stages of the Ghanaian academic trajectory. This multilingual position of Ghana makes it common to find elements of CS present in FL classrooms, where both instructors and learners occasionally resort to English for instructional conversations (Chachu 2016). This study focuses on university students’ perceptions of CS between English as the source language and French as a foreign language (FFL) during the instruction of the latter. Some frameworks for teaching and learning FFL in Ghana prefer that the language of instruction and interaction be limited to the target language only (Soku and Azanku 2015). The Common European Framework of References for Languages (CEFR) however, suggests the possible use of CS for purposes of instruction when there are favourable conditions (Council of Europe 2001). To identify the roles of CS and investigate FL students’ opinions on this, our data was gathered from classroom observations and questionnaires that had both Likert scale and open-ended questions. This was at the end of their first semester of studying university-level FFL. It was observed that CS plays a significant role in the understanding of linguistic concepts and greatly improves students’ engagement and participation in the FL class. Of the 36 respondents, 34 felt their academic needs were met when the language of instruction and interaction alternated between French and English. Twenty-nine students reported losing focus easily in the FFL class in instances where the medium of interaction was limited to French alone, while 26 reported feeling nervous whenever they were asked to only communicate in French. For metalinguistic aspects of the language, 33 students, constituting 91.7% were of the view that a careful blend of English and French could go a long way to aid their understanding and foster the learning of grammatical concepts. Generally, students attested to better retention of vocabulary in the target language when there was a good blend of both languages. It is recommended therefore that language instructors resort periodically to CS to ensure understanding and engagement from students. CS also helps learners have the opportunity to intervene with their limited linguistic structures when they are faced with linguistic challenges in the FL classroom. Click here to read the corresponding ISSOTL blog post.

Modern data-driven spoken language systems (SLS) require manual semantic annotation for training spoken language understanding parsers. Multilingual porting of SLS demands significant manual effort and language resources, as this manual annotation has to be replicated. Crowdsourcing is an accessible and cost-effective alternative to traditional methods of collecting and annotating data. The application of crowdsourcing to simple tasks has been well investigated. However, complex tasks, like cross-language semantic annotation transfer, may generate low judgment agreement and/or poor performance. The most serious issue in cross-language porting is the absence of reference annotations in the target language; thus, crowd quality control and the evaluation of the collected annotations is difficult. In this paper we investigate targeted crowdsourcing for semantic annotation transfer that delegates to crowds a complex task such as segmenting and labeling of concepts taken from a domain ontology; and evaluation using source language annotation. To test the applicability and effectiveness of the crowdsourced annotation transfer we have considered the case of close and distant language pairs: Italian–Spanish and Italian–Greek. The corpora annotated via crowdsourcing are evaluated against source and target language expert annotations. We demonstrate that the two evaluation references (source and target) highly correlate with each other; thus, drastically reduce the need for the target language reference annotations.

The lack of terminology and language resources for the under-resourced South African languages poses a serious problem for effective communication in specialised fields such as law, education, health, agriculture, science, and technology. The ability to use all South African languages in all contemporary fields requires the existence of relevant terminology and resources. The article aims to semi-automatically identify and extract terminology for creating an English, Xitsonga, Siswati, and isiNdebele quadrilingual dictionary. Given parallel texts in the four different languages, we use ParaConc to identify and extract terminology in one language and the corresponding translations in the other languages. In this study, English is used as the source language, while Xitsonga, Siswati, and isiNdebele are the target languages. This process allowed us to identify specific lexical items in the source language (manually) and their translation equivalents in the target languages (automatically). The result was a collection of extracted terminology lists that can be used to compile a specialised quadrilingual dictionary for English and three of the under-resourced languages. We show the usefulness of ParaConc to semi-automatically extract quadrilingual terminology lists, which, by creating quadrilingual dictionaries, will contribute to the development and promotion of multilingualism in South Africa.