Language Model Supervision for Handwriting Recognition Model Adaptation

To better tackle the named entity recognition (NER) problem on languages with little/no labeled data, cross-lingual NER must effectively leverage knowledge learned from source languages with rich labeled data. Previous works on cross-lingual NER are mostly based on label projection with pairwise texts or direct model transfer. However, such methods either are not applicable if the labeled data in the source languages is unavailable, or do not leverage information contained in unlabeled data in the target language. In this paper, we propose a teacher-student learning method to address such limitations, where NER models in the source languages are used as teachers to train a student model on unlabeled data in the target language. The proposed method works for both single-source and multi-source cross-lingual NER. For the latter, we further propose a similarity measuring method to better weight the supervision from different teacher models. Extensive experiments for 3 target languages on benchmark datasets well demonstrate that our method outperforms existing state-of-the-art methods for both single-source and multi-source cross-lingual NER.

Since the quality of the language model directly affects the performance of the spoken dialog system (SDS), we should use a statistical language model (LM) trained with a large amount of data that is matched to the task domain. When porting a SDS to another language, however, it is costly to re-collect a large amount of user utterances in the target language. We thus use the language resources in a source language by utilizing statistical machine translation. The main challenge in this work is to induct automatic speech recognition results collected using a speech-input system that differs from the target SDS both in the task and the target language. To select appropriate sentences to be included in the training data for the LM, we induct a spoken language understanding module of the dialog system in the source language. Experimental construction using over three million user utterances showed that it is vital to conduct a selection from the translation results.

Over the past few years, Large Language Models of Code (Code LLMs) have started to have a significant impact on programming practice. Code LLMs are also emerging as building blocks for research in programming languages and software engineering. However, the quality of code produced by a Code LLM varies significantly by programming language. Code LLMs produce impressive results on high-resource programming languages that are well represented in their training data (e.g., Java, Python, or JavaScript), but struggle with low-resource languages that have limited training data available (e.g., OCaml, Racket, and several others). This paper presents an effective approach for boosting the performance of Code LLMs on low-resource languages using semi-synthetic data. Our approach, called M ulti PL-T, generates high-quality datasets for low-resource languages, which can then be used to fine-tune any pretrained Code LLM. M ulti PL-T translates training data from high-resource languages into training data for low-resource languages in the following way. 1) We use a Code LLM to synthesize unit tests for commented code from a high-resource source language, filtering out faulty tests and code with low test coverage. 2) We use a Code LLM to translate the code from the high-resource source language to a target low-resource language. This gives us a corpus of candidate training data in the target language, but many of these translations are wrong. 3) We use a lightweight compiler to compile the test cases generated in (1) from the source language to the target language, which allows us to filter our obviously wrong translations. The result is a training corpus in the target low-resource language where all items have been validated with test cases. We apply this approach to generate tens of thousands of new, validated training items for five low-resource languages: Julia, Lua, OCaml, R, and Racket, using Python as the source high-resource language. Furthermore, we use an open Code LLM (StarCoderBase) with open training data (The Stack), which allows us to decontaminate benchmarks, train models without violating licenses, and run experiments that could not otherwise be done. Using datasets generated with M ulti PL-T, we present fine-tuned versions of StarCoderBase and Code Llama for Julia, Lua, OCaml, R, and Racket that outperform other fine-tunes of these base models on the natural language to code task. We also present Racket fine-tunes for two very recent models, DeepSeek Coder and StarCoder2, to show that M ulti PL-T continues to outperform other fine-tuning approaches for low-resource languages. The M ulti PL-T approach is easy to apply to new languages, and is significantly more efficient and effective than alternatives such as training longer.

We study a new problem of cross-lingual transfer learning for event coreference resolution (ECR) where models trained on data from a source language are adapted for evaluations in different target languages. We introduce the first baseline model for this task based on XLM-RoBERTa, a state-of-the-art multilingual pre-trained language model. We also explore language adversarial neural networks (LANN) that present language discriminators to distinguish texts from the source and target languages to improve the language generalization for ECR. In addition, we introduce two novel mechanisms to further enhance the general representation learning of LANN, featuring: (i) multi-view alignment to penalize cross coreference-label alignment of examples in the source and target languages, and (ii) optimal transport to select close examples in the source and target languages to provide better training signals for the language discriminators. Finally, we perform extensive experiments for cross-lingual ECR from English to Spanish and Chinese to demonstrate the effectiveness of the proposed methods.

The present study proposes a source speech register analysis model that incorporates textual and situational components and parameters of register equivalence in the target language to identify the relations between the source language register and student interpreter’s performance, textual register mismatch between the source and target languages, and the relevance between register equivalence parameters in the target language and interpreting quality. The findings are as follows: first, source language register does have influence on interpreter’s performance, and relations exist between the two. In general, higher source language register brings better interpreting quality. Three assessment criteria and the overall quality differ across different levels of registers in the source language. Second, register mismatch between the source and target languages happens since interpreters are inclined to lower the register of interpreting output from the high-register source speech and increase the register level in the target language in the case of the low-register speech. Third, intonation and voice have the highest correlations with the interpreters’ performance. Fluency ranks the second, and lexical-semantic choices the third.

Though designing of classifies for Indic script handwriting recognition has been researched with enough attention, use of language model has so far received little exposure. This paper attempts to develop a weighted finite-state transducer (WFST) based language model for improving the current recognition accuracy. Both the recognition hypothesis (i.e. the segmentation lattice) and the lexicon are modeled as two WFSTs. Concatenation of these two FSTs accept a valid word(s) which is (are) present in the recognition lattice. A third FST called error FST is also introduced to retrieve certain words which were missing in the previous concatenation operation. The proposed model has been tested for online Bangla handwriting recognition though the underlying principle can equally be applied for recognition of offline or printed words. Experiment on a part of ISI-Bangla handwriting database shows that while the present classifiers (without using any language model) can recognize about 73% word, use of recognition and lexicon FSTs improve this result by about 9% giving an average word-level accuracy of 82%. Introduction of error FST further improves this accuracy to 93%. This remarkable improvement in word recognition accuracy by using FST-based language model would serve as a significant revelation for the research in handwriting recognition, in general and Indic script handwriting recognition, in particular.

The study focuses on teaching the translation of audiovisual memes: comic image/video and text combinations that tend to go viral on the Internet. The key features of a meme are a comic effect resulting from surprise, disrupted expectations, the clash between the text and the video; “viral” expansion across the Internet and other social media. Mastering the skills required to translate such audiovisual content is essential since memes are highly popular on the Internet with all age groups, there are numerous allusions references to the popular memes in the mass media, fiction, movies, and TV series. Teaching audiovisual meme translation shall consist of analyzing the meme properties, building meme comprehension and metacognitive analysis skills, and mastering specific approaches and techniques for meme representation in the source and target languages. An emphasis should be placed on developing the ability to adequately reconstruct meme ecosystems in the source language and find their analogs in the target language as well as the skills required to identify archetypical narratives in the source language and build them in the target language. It is shown that memes have some similarities to set expressions and proverbs. The meme translation curriculum shall build the ability to find and translate set expressions and proverbs in audiovisual works. It is also shown that translation teachers should have specialized training in teaching the translation of memes as a special type of audiovisual content. It can be implemented in the format of short seminars/webinars, or within the teacher further education framework.

Multi-Language Speech-to-Text Translation (ST) plays a pivotal role in bridging linguistic barriers by converting spoken language into written text across different languages.This project aims to develop a robust ST model tailored for low-resource Indian languages, specifically targeting the Indo-Aryan and Dravidian language families.The dataset used consists of speeches from conferences and TED Talks, along with their corresponding transcriptions in English (source language) and translations in Hindi, Bengali, and Tamil (target languages).By tackling the lack of data and disparities in attention within low-resource languages, the paper strives to create an efficient ST system capable of real-world deployment.Additionally, existing resources in related languages are leveraged and word-level translation resources are explored to enhance translation accuracy.

The current state-of-the-art for few-shot cross-lingual transfer learning first trains on abundant labeled data in the source language and then fine-tunes with a few examples on the target language, termed target-adapting. Though this has been demonstrated to work on a variety of tasks, in this paper we show some deficiencies of this approach and propose a one-step mixed training method that trains on both source and target data with stochastic gradient surgery, a novel gradient-level optimization. Unlike the previous studies that focus on one language at a time when target-adapting, we use one model to handle all target languages simultaneously to avoid excessively language-specific models. Moreover, we discuss the unreality of utilizing large target development sets for model selection in previous literature. We further show that our method is both development-free for target languages, and is also able to escape from overfitting issues. We conduct a large-scale experiment on 4 diverse NLP tasks across up to 48 languages. Our proposed method achieves state-of-the-art performance on all tasks and outperforms target-adapting by a large margin, especially for languages that are linguistically distant from the source language, e.g., 7.36% F1 absolute gain on average for the NER task, up to 17.60% on Punjabi.

This paper discusses English-poetry translation, Heusca by John Conford into Indonesian translated by Chairil Anwar. The purpose of this study is to determine how the translator considers aesthetic and expressive values from the source language and translate them into target language. Aesthetic values are the means of delivery thought through the use of poetic language such as diction or word choice, metaphor, imageri, and others figurative language. Meanwhile, the expressive values are the means of delivery thoughts through the author's mind or emotions such as structure, rhyme, and pronunciation. The method used in this study is a qualitative descriptive analysis of the meaning. The results of analysis show that in terms of meaning, aesthetic values, and structures, Heusca is the result of a semantic translation and faithful to source language. Translator managed to retain the meaning of source language. In other word, the translation can meet the requirements of fidelity. In terms of expressive values in particular aspects of meter and rhyme are not so faithful to source language because the source language and the target language have some differences. Basically, this is acceptable that the Indonesian language as target language has a very big difference in terms of pronunciation with English as the source language. As a result, meter and rhyme of target language may not be forced into the source language without damaging the naturalness of meaning for target language readers. It can be said that the translation can meet the transparency criteria because source language appeared in poetry comes naturally to native speakers of English.

The ongoing debate on code switching (CS) in multilingual societies and Foreign Language (FL) classes is still relevant and worth considering. This linguistic phenomenon simply refers to altering from one language to another during speech and writing (Richards and Schmidt 2013). While some authorities propose that interactions between learners and instructors in the FL class be carried out only in the target language, other opinions favour a careful blend of both target and source languages. Ghana, a multilingual country in West Africa, adopted English as its official language during the colonial era. French was later introduced to promote regional integration among Ghana and her neighbours, especially since Ghana is greatly surrounded by francophone countries within the subregion (Gandolfi et al. 2021). French has since then been the second most widely taught FL in the country, with a presence in almost all stages of the Ghanaian academic trajectory. This multilingual position of Ghana makes it common to find elements of CS present in FL classrooms, where both instructors and learners occasionally resort to English for instructional conversations (Chachu 2016). This study focuses on university students’ perceptions of CS between English as the source language and French as a foreign language (FFL) during the instruction of the latter. Some frameworks for teaching and learning FFL in Ghana prefer that the language of instruction and interaction be limited to the target language only (Soku and Azanku 2015). The Common European Framework of References for Languages (CEFR) however, suggests the possible use of CS for purposes of instruction when there are favourable conditions (Council of Europe 2001). To identify the roles of CS and investigate FL students’ opinions on this, our data was gathered from classroom observations and questionnaires that had both Likert scale and open-ended questions. This was at the end of their first semester of studying university-level FFL. It was observed that CS plays a significant role in the understanding of linguistic concepts and greatly improves students’ engagement and participation in the FL class. Of the 36 respondents, 34 felt their academic needs were met when the language of instruction and interaction alternated between French and English. Twenty-nine students reported losing focus easily in the FFL class in instances where the medium of interaction was limited to French alone, while 26 reported feeling nervous whenever they were asked to only communicate in French. For metalinguistic aspects of the language, 33 students, constituting 91.7% were of the view that a careful blend of English and French could go a long way to aid their understanding and foster the learning of grammatical concepts. Generally, students attested to better retention of vocabulary in the target language when there was a good blend of both languages. It is recommended therefore that language instructors resort periodically to CS to ensure understanding and engagement from students. CS also helps learners have the opportunity to intervene with their limited linguistic structures when they are faced with linguistic challenges in the FL classroom. Click here to read the corresponding ISSOTL blog post.

The lack of terminology and language resources for the under-resourced South African languages poses a serious problem for effective communication in specialised fields such as law, education, health, agriculture, science, and technology. The ability to use all South African languages in all contemporary fields requires the existence of relevant terminology and resources. The article aims to semi-automatically identify and extract terminology for creating an English, Xitsonga, Siswati, and isiNdebele quadrilingual dictionary. Given parallel texts in the four different languages, we use ParaConc to identify and extract terminology in one language and the corresponding translations in the other languages. In this study, English is used as the source language, while Xitsonga, Siswati, and isiNdebele are the target languages. This process allowed us to identify specific lexical items in the source language (manually) and their translation equivalents in the target languages (automatically). The result was a collection of extracted terminology lists that can be used to compile a specialised quadrilingual dictionary for English and three of the under-resourced languages. We show the usefulness of ParaConc to semi-automatically extract quadrilingual terminology lists, which, by creating quadrilingual dictionaries, will contribute to the development and promotion of multilingualism in South Africa.

Sentiment analysis in various languages has been a research hotspot with many applications. However, sentiment resources (e.g., labeled corpora, sentiment lexicons) of different languages are unbalanced in terms of quality and quantity, which arouses interests in cross-lingual sentiment analysis aiming at using the resources in a source language to improve sentiment analysis in a target language. Nevertheless, many existing cross-lingual related works rely on a certain machine translation system to directly adapt the labeled data from the source language to the target language, which usually suffers from inaccurate results generated by the machine translation system. On the other hand, most sentiment analysis studies focus on document-level sentiment classification that cannot solve the aspect dependency problem of sentiment words. For instance, in the reviews on a cell phone, long is positive for the lifespan of its battery, but negative for the response time of its operating system. To solve these problems, this paper develops a novel Cross-Lingual Joint Aspect/Sentiment (CLJAS) model to carry out aspect-specific sentiment analysis in a target language using the knowledge learned from a source language. Specifically, the CLJAS model jointly detects aspects and sentiments of two languages simultaneously by incorporating sentiments into a cross-lingual topic model framework. Extensive experiments on different domains and different languages demonstrate that the proposed model can significantly improve the accuracy of sentiment classification in the target language.

In this paper, we describe the work developed in the creation of syllable-based language model for continuous speech recognition system for Myanmar language. Speech recognition systems contain language model as the part of prediction word order sequence. In English and other languages, speech recognition system can use word-based language model. Since Myanmar is monosyllabic and syllable-timed language, the syllable-based language model is more suitable in speech recognition system. At the first step, this paper explains the structure of traditional (Phrase based) language model and describes how to normalized Myanmar sentences and building of normalized Myanmar language model. In the second step, the evaluation of the two language models is expressed. Finally, the comparison of these two language models with the perplexity values is described. The syllable-based Myanmar language model has lower perplexity value than the traditional language model. This language model can support the creation of word-based Myanmar language model.

Modern data-driven spoken language systems (SLS) require manual semantic annotation for training spoken language understanding parsers. Multilingual porting of SLS demands significant manual effort and language resources, as this manual annotation has to be replicated. Crowdsourcing is an accessible and cost-effective alternative to traditional methods of collecting and annotating data. The application of crowdsourcing to simple tasks has been well investigated. However, complex tasks, like cross-language semantic annotation transfer, may generate low judgment agreement and/or poor performance. The most serious issue in cross-language porting is the absence of reference annotations in the target language; thus, crowd quality control and the evaluation of the collected annotations is difficult. In this paper we investigate targeted crowdsourcing for semantic annotation transfer that delegates to crowds a complex task such as segmenting and labeling of concepts taken from a domain ontology; and evaluation using source language annotation. To test the applicability and effectiveness of the crowdsourced annotation transfer we have considered the case of close and distant language pairs: Italian–Spanish and Italian–Greek. The corpora annotated via crowdsourcing are evaluated against source and target language expert annotations. We demonstrate that the two evaluation references (source and target) highly correlate with each other; thus, drastically reduce the need for the target language reference annotations.