A Study of Machine Translation Models for Kannada-Tulu

In the modern era, the field of Machine Translation (MT) has seen a significant shift towards Neural Machine Translation (NMT) techniques, which have surpassed traditional Statistical Machine Translation (SMT) models in terms of the quality of translation. Despite this, the efficacy of these techniques may differ based on the language combination in consideration. While SMT is somewhat more flexible in this regard, NMT often needs sizable parallel corpora to attain high translation accuracy. As a result, a benchmark system capable of offering sufficient translation for languages with limited resources, like Nepali, remains a pipe dream. This paper focuses on translating text using statistical and neural MT techniques for the under-resourced English-Nepali language pair. As a part of this system development, we built a parallel corpus of English-Nepali in the tourism domain. We explore the impact of different tokenization techniques on translation outcomes. A substantial analysis is also done for the performance of both approaches using automatic evaluation metrics, BLEU and TER. This paper aims to provide insights into the applicability of SMT and NMT for the under-resourced English-Nepali language pair in light of two popular epitomes of tokenization and to determine the most effective approach for achieving accurate translations.

Applying convolutional attention mechanisms and Human Memory Search for effective English-Urdu translation

Neural machine translation (NMT) is a remarkable approach which performs much better than the Statistical machine translation (SMT) models when there is an abundance of parallel corpus. However, vanilla NMT is primarily based upon word-level with a fixed vocabulary. Therefore, low resource morphologically rich languages such as Sinhala are mostly affected by the out of vocabulary (OOV) and Rare word problems. Recent advancements in subword techniques have opened up opportunities for low resource communities by enabling open vocabulary translation. In this paper, we extend our recently published state-of-the-art EN-SI translation system using the transformer and explore standard subword techniques on top of it to identify which subword approach has a greater effect on English Sinhala language pair. Our models demonstrate that subword segmentation strategies along with the state-of-the-art NMT can perform remarkably when translating English sentences into a rich morphology language regardless of a large parallel corpus.

Sanskrit is one of the earliest native languages and is correctly described as "the gods' language" because of its wide use in Indian religious literature from the past. However, it is becoming less popular in modern India. Due in significant part to the need for more materials for translation both in and out of Sanskrit, it is no longer commonly utilized. This study explores the feasibility of using machine translation (MT) to provide a link between Sanskrit and, one of the earliest native languages, and its contemporary descendant Hindi. A study was conducted between existing modelling methodologies, notably Statistical machine translation (SMT), and the proposed novel deep learning-based Machine translation strategy using a manually created parallel corpus for the Sanskrit-Hindi language pair. While SMT creates interpretations by mapping phrases from the languages of the source and destination, statistical models, and bilingual text corpora for learning parameters, neural machine translation (NMT) frequently models entire phrases in a single integrated model, using a convolutional neural network to calculate the probability of a word sequence. The proposed NMT model is implemented using an encoder-decoder with an attention mechanism paradigm and the inclusion of gated recurrent units. Our approach involved development of a novel model for Sanskrit-Hindi machine translation using deep learning and the creation of parallel corpora for the Sanskrit-Hindi language pair. The proposed model is evaluated on automated and human-based metrics, and results show that our proposed deep learning-based model outperforms statistical modelling techniques on Moses, surpassing them both with a BLEU score of 53.8% compared to 34.56%. This article examines the undiscovered area of machine translation from Sanskrit to Hindi and discusses the main benefits and drawbacks of statistical and neural machine translation while providing a fresh viewpoint on the subject.

Machine translation aims to minimize the language barrier between people of different linguistic backgrounds. Machine translation is an automatic translation technique between pairs of different languages. Machine translation using neural network or neural machine translation came into the picture due to several limitation associated with its predecessors which are rule based and statistical based models. For large data sets with a rich range of vocabulary, the neural network machine translation system provides fair translation accuracy. We have observed that there remain very few machine translators dedicated to Indian languages, especially those spoken in the North-East area. So our paper mainly focuses on implementing a neural machine translation system for the English-Assamese language. In this paper we used five different neural machine translation models for English-Assamese language pair using LSTMs and GRUs, along with an attention layer. We have made a comparison analysis based on the performance results of these models. We used BLEU Score to calculate the accuracy of these five models, thus achieving a BLEU score of 34.168%, which was the highest among the five models.

Machine translation is a computer-based translation process that receives a set of words from a particular humanreadable language as input and outputs a second set of words in the intended human-readable language. Machine translation has assisted linguists and sociologists all across the world. A machine translation model may be statistical, rule-based, or based on neural networks. The Neural Machine Translation (NMT) model was introduced in response to the numerous drawbacks of rule- based and statistical-based machine translation models. This study adds a parallel corpus of Assamese and English and develops the NMT system, a transformer model with a self- attention mechanism. For translation from English to Assamese, the system received a higher bilingual evaluation understudy (BLEU) score of 7.6, and for translation from Assamese to English,a BLEU score of 23.2.

Compared to traditional statistical machine translation (SMT), such as phrase-based machine translation (PBMT), neural machine translation (NMT) often sacrifices adequacy for the sake of fluency. We propose a method to combine the advantages of traditional SMT and NMT by exploiting an existing phrase-based SMT model to compute the phrase-based decoding cost for an NMT output and then using this cost to rerank the n-best NMT outputs. The main challenge in implementing this approach is that NMT outputs may not be in the search space of the standard phrase-based decoding algorithm, because the search space of PBMT is limited by the phrase-based translation rule table. We propose a phrase-based soft forced decoding algorithm, which can always successfully find a decoding path for any NMT output. We show that using the phrase-based decoding cost to rerank the NMT outputs can successfully improve translation quality on four different language pairs.

Corpus-based machine translation (MT) has been the main approach to developing and implementing MT systems in both academia and the industry over the last three decades. In this field, the type and size of the corpus used for training MT engines have presented problems for both statistical MT (SMT) systems as well as neural MT (NMT) systems, being the two dominant corpusbased approaches. Moreover, language pairs such as Turkish-English have been understudied within this framework. This article aims to evaluate the translation quality in Turkish-to-English custom MT systems that have been trained on different corpus sizes and types. Two NMT engines and two SMT engines were trained on the KantanMT platform using two different training corpus types with either only domain-specific cardiology corpus or this corpus plus a mixed-domain corpus. The study conducted both automatic evaluations with metrics including BLEU, F-Measure and TER, as well as a comprehensive human evaluation with metrics including fluency, A/B test, and adequacy. Lastly, the study realized a separate, subjective terminology evaluation in order to investigate how differently MT systems handle terminology, as this is a crucial aspect for specific-domain text types such as cardiology. While the automatic evaluation results suggest the SMT engines to perform better than NMT engines, all human evaluators rated the mixed-domain NMT engine as the highest performing one. However, the terminology evaluation task demonstrated SMT to still be able to perform better and to commit less terminology errors, despite the industry and academia shifting toward NMT engines.

We conduct the first experiment in the literature in which a novel is translated automatically and then post-edited by professional literary translators. Our case study is Warbreaker, a popular fantasy novel originally written in English, which we translate into Catalan. We translated one chapter of the novel (over 3,700 words, 330 sentences) with two data-driven approaches to Machine Translation (MT): phrase-based statistical MT (PBMT) and neural MT (NMT). Both systems are tailored to novels; they are trained on over 100 million words of fiction. In the post-editing experiment, six professional translators with previous experience in literary translation translate subsets of this chapter under three alternating conditions: from scratch (the norm in the novel translation industry), post-editing PBMT, and post-editing NMT. We record all the keystrokes, the time taken to translate each sentence, as well as the number of pauses and their duration. Based on these measurements, and using mixed-effects models, we study post-editing effort across its three commonly studied dimensions: temporal, technical and cognitive. We observe that both MT approaches result in increases in translation productivity: PBMT by 18%, and NMT by 36%. Post-editing also leads to reductions in the number of keystrokes: by 9% with PBMT, and by 23% with NMT. Finally, regarding cognitive effort, post-editing results in fewer (29 and 42% less with PBMT and NMT, respectively) but longer pauses (14 and 25%).

Neural Machine Translation (NMT) performs sophisticated technology of translation between particular languages accurately and perfectly compared with traditional Statistical Machine Translation (SMT). Current models like Transformer and NMT can produce better translation results. In order to create NMT, we need a large-scale parallel corpus for training and validation of NMT data. Existing parallel corpus such as WMT English-German or WMT English-France become benchmark dataset or tasks in WMT. Conversely, ethnic language parallel corpus like English-to-Sundanese ethnic language is rarely used. This paper presents the collection and analysis of large-scale English-to-Sundanese parallel corpus. Our Parallel corpus encompasses various topics like sports, economics, politics and others. We produced a BLEU score English-to-Sundanese 38.91 that indicated our corpus valid.

Neural machine translation is a machine translation system that uses artificial neural networks to identify nonlinear relationships between bilingual sentence pairs. The language English-Kannada pair has met with less attention in the field of Neural Machine Translation (NMT), mostly due to lack of parallel corpora and the complexity of Kannada's linguistic structure. This research proposes a novel NMT model based on Gated Recurrent Units (GRU) and a Hybrid attention mechanism designed specifically for Kannada's morphological complexity and compared with existing models. By adding language-specific preprocessing techniques and employing data augmentation tactics, our model outperforms previous LSTM, GRU and Bi-LSTM-based algorithms in terms of BLEU and METEOR scores. The proposed model with hybrid attention might be a useful substitute in low-resource settings because transformers and other large models require more resources. The comparative study indicates that the proposed methodology improves the translation by 10% increase in Bilingual Evaluation Understudy (BLEU) and accuracy of 80% and achieved a Rouge Score of 0.9.

Machine translation (MT) models usually translate a text at sentence level by considering isolated sentences, which is based on a strict assumption that the sentences in a text are independent of one another.However, the fact is that the texts at discourse level have properties going beyond individual sentences.These properties reveal texts in the frequency and distribution of words, word senses, referential forms and syntactic structures.Disregarding dependencies across sentences will harm translation quality especially in terms of coherence, cohesion, and consistency.To solve these problems, several approaches have previously been investigated for conventional statistical machine translation (SMT).With the fast growth of neural machine translation (NMT), discourse-level NMT has drawn increasing attention from researchers.In this work, we review major works on addressing discourse related problems for both SMT and NMT models with a survey of recent trends in the fields.

In Brazil, the governmental body responsible for overseeing and coordinating post-graduate programs, CAPES, keeps records of all theses and dissertations presented in the country. Information regarding such documents can be accessed online in the Theses and Dissertations Catalog (TDC), which contains abstracts in Portuguese and English, and additional metadata. Thus, this database can be a potential source of parallel corpora for the Portuguese and English languages. In this article, we present the development of a parallel corpus from TDC, which is made available by CAPES under the open data initiative. Approximately 240,000 documents were collected and aligned using the Hunalign tool. We demonstrate the capability of our developed corpus by training Statistical Machine Translation (SMT) and Neural Machine Translation (NMT) models for both language directions, followed by a comparison with Google Translate (GT). Both translation models presented better BLEU scores than GT, with NMT system being the most accurate one. Sentence alignment was also manually evaluated, presenting an average of 82.30% correctly aligned sentences. Our parallel corpus is freely available in TMX format, with complementary information regarding document metadata

Machine Translation (MT) has recently been strengthened by the emergence of deep neural network techniques, which have brought the dream of language translation in community-based practices and clinical practices closer to reality. This chapter introduces the clinical practice of MT, especially in low resource languages. The following are some of the highlights of this chapter. How MT played a crucial role in good social reasons is focused on in this chapter. The chapter starts with the history of translation technologies that have played a vital role in various crisis and relief scenarios as the Haitian earthquake in 2010 and Translators without Borders (TWB), respectively. The Standby Task Force (SBTF) deployed the technology to tackle misinformation during Coronavirus Pandemic 2019. Next, the chapter explores the importance of the MT system, where it describes MT literacy as required across society to impact the healthcare of human beings. It describes the challenges of translation for clinical texts. Some of those are highlighted as (a) Domain: characteristics of clinical text differentiated from texts with other domains, (b) code mixed low-resource languages: noisy clinical data and lower quantity of data in the digital form available for computation, (c) lack of clinical domain corpora, and (d) very rich morphological structure: e.g., procedures (e.g., hepatico-cholangio-jejuno-stomy) and chemicals (e.g., Hydro-xy-nitro-di-hydro-thy-mine). Strategies or approaches to building MT on the clinical text can be categorized as: (1) Rule-based MT (early days), (2) Statistical MT (SMT, one of the successful methods in MT history), and (3) Neural MT (NMT, recent advancements of MT). Both SMT and NMT that work on same grounds by learning from text and are the primary focus of this chapter. L earning transparency for the patient's use of clinical data is an essential component. Much of today's healthcare data is unstructured text, such as clinical text, doctor's notes, nursing notes, lab reports, test reports, and more. A clinical practitioner is faced with a language barrier for communicating with a patient to interpret patient histories, examine a clinical diagnosis, or restate the recommended treatment method and follow-up to aid comprehension. The only way a medical practitioner encourages patients to ask questions or respond to queries is by directing them to input text with the help of MT. Finally, the chapter discusses contemporary techniques with Neural Machine Translation (NMT), namely multilingual, transfer learning on clinical data.

The rapid spread of fake news has become a critical issue in the digital era, posing challenges to information authenticity and decision-making. This research presents a novel method for fake news detection leveraging a Generative Adversarial Network (GAN) combined with the BLEU (Bilingual Evaluation Understudy) score for evaluating textual quality. The proposed model uses a GAN framework to generate synthetic news data and trains a classifier to distinguish between genuine and fabricated articles. The BLEU score, commonly used in machine translation, is adapted to assess the accuracy of generated text against real-world news. Experimental results show the effectiveness of this approach, with detection performance categorized based on BLEU score ranges: Excellent (0.7+), Good (0.5-0.7), Average (0.3-0.5), and Poor (<0.3). With a BLEU score of 0.66, the model demonstrates strong performance in distinguishing fake from real news, with potential applications in automated content moderation and misinformation detection.