Machine Translation Systems Based on Classical-Statistical-Deep-Learning Approaches

Sanskrit is one of the earliest native languages and is correctly described as "the gods' language" because of its wide use in Indian religious literature from the past. However, it is becoming less popular in modern India. Due in significant part to the need for more materials for translation both in and out of Sanskrit, it is no longer commonly utilized. This study explores the feasibility of using machine translation (MT) to provide a link between Sanskrit and, one of the earliest native languages, and its contemporary descendant Hindi. A study was conducted between existing modelling methodologies, notably Statistical machine translation (SMT), and the proposed novel deep learning-based Machine translation strategy using a manually created parallel corpus for the Sanskrit-Hindi language pair. While SMT creates interpretations by mapping phrases from the languages of the source and destination, statistical models, and bilingual text corpora for learning parameters, neural machine translation (NMT) frequently models entire phrases in a single integrated model, using a convolutional neural network to calculate the probability of a word sequence. The proposed NMT model is implemented using an encoder-decoder with an attention mechanism paradigm and the inclusion of gated recurrent units. Our approach involved development of a novel model for Sanskrit-Hindi machine translation using deep learning and the creation of parallel corpora for the Sanskrit-Hindi language pair. The proposed model is evaluated on automated and human-based metrics, and results show that our proposed deep learning-based model outperforms statistical modelling techniques on Moses, surpassing them both with a BLEU score of 53.8% compared to 34.56%. This article examines the undiscovered area of machine translation from Sanskrit to Hindi and discusses the main benefits and drawbacks of statistical and neural machine translation while providing a fresh viewpoint on the subject.

Machine translation aims to minimize the language barrier between people of different linguistic backgrounds. Machine translation is an automatic translation technique between pairs of different languages. Machine translation using neural network or neural machine translation came into the picture due to several limitation associated with its predecessors which are rule based and statistical based models. For large data sets with a rich range of vocabulary, the neural network machine translation system provides fair translation accuracy. We have observed that there remain very few machine translators dedicated to Indian languages, especially those spoken in the North-East area. So our paper mainly focuses on implementing a neural machine translation system for the English-Assamese language. In this paper we used five different neural machine translation models for English-Assamese language pair using LSTMs and GRUs, along with an attention layer. We have made a comparison analysis based on the performance results of these models. We used BLEU Score to calculate the accuracy of these five models, thus achieving a BLEU score of 34.168%, which was the highest among the five models.

In the modern era, the field of Machine Translation (MT) has seen a significant shift towards Neural Machine Translation (NMT) techniques, which have surpassed traditional Statistical Machine Translation (SMT) models in terms of the quality of translation. Despite this, the efficacy of these techniques may differ based on the language combination in consideration. While SMT is somewhat more flexible in this regard, NMT often needs sizable parallel corpora to attain high translation accuracy. As a result, a benchmark system capable of offering sufficient translation for languages with limited resources, like Nepali, remains a pipe dream. This paper focuses on translating text using statistical and neural MT techniques for the under-resourced English-Nepali language pair. As a part of this system development, we built a parallel corpus of English-Nepali in the tourism domain. We explore the impact of different tokenization techniques on translation outcomes. A substantial analysis is also done for the performance of both approaches using automatic evaluation metrics, BLEU and TER. This paper aims to provide insights into the applicability of SMT and NMT for the under-resourced English-Nepali language pair in light of two popular epitomes of tokenization and to determine the most effective approach for achieving accurate translations.

We conduct the first experiment in the literature in which a novel is translated automatically and then post-edited by professional literary translators. Our case study is Warbreaker, a popular fantasy novel originally written in English, which we translate into Catalan. We translated one chapter of the novel (over 3,700 words, 330 sentences) with two data-driven approaches to Machine Translation (MT): phrase-based statistical MT (PBMT) and neural MT (NMT). Both systems are tailored to novels; they are trained on over 100 million words of fiction. In the post-editing experiment, six professional translators with previous experience in literary translation translate subsets of this chapter under three alternating conditions: from scratch (the norm in the novel translation industry), post-editing PBMT, and post-editing NMT. We record all the keystrokes, the time taken to translate each sentence, as well as the number of pauses and their duration. Based on these measurements, and using mixed-effects models, we study post-editing effort across its three commonly studied dimensions: temporal, technical and cognitive. We observe that both MT approaches result in increases in translation productivity: PBMT by 18%, and NMT by 36%. Post-editing also leads to reductions in the number of keystrokes: by 9% with PBMT, and by 23% with NMT. Finally, regarding cognitive effort, post-editing results in fewer (29 and 42% less with PBMT and NMT, respectively) but longer pauses (14 and 25%).

Machine Translation (MT) has recently been strengthened by the emergence of deep neural network techniques, which have brought the dream of language translation in community-based practices and clinical practices closer to reality. This chapter introduces the clinical practice of MT, especially in low resource languages. The following are some of the highlights of this chapter. How MT played a crucial role in good social reasons is focused on in this chapter. The chapter starts with the history of translation technologies that have played a vital role in various crisis and relief scenarios as the Haitian earthquake in 2010 and Translators without Borders (TWB), respectively. The Standby Task Force (SBTF) deployed the technology to tackle misinformation during Coronavirus Pandemic 2019. Next, the chapter explores the importance of the MT system, where it describes MT literacy as required across society to impact the healthcare of human beings. It describes the challenges of translation for clinical texts. Some of those are highlighted as (a) Domain: characteristics of clinical text differentiated from texts with other domains, (b) code mixed low-resource languages: noisy clinical data and lower quantity of data in the digital form available for computation, (c) lack of clinical domain corpora, and (d) very rich morphological structure: e.g., procedures (e.g., hepatico-cholangio-jejuno-stomy) and chemicals (e.g., Hydro-xy-nitro-di-hydro-thy-mine). Strategies or approaches to building MT on the clinical text can be categorized as: (1) Rule-based MT (early days), (2) Statistical MT (SMT, one of the successful methods in MT history), and (3) Neural MT (NMT, recent advancements of MT). Both SMT and NMT that work on same grounds by learning from text and are the primary focus of this chapter. L earning transparency for the patient's use of clinical data is an essential component. Much of today's healthcare data is unstructured text, such as clinical text, doctor's notes, nursing notes, lab reports, test reports, and more. A clinical practitioner is faced with a language barrier for communicating with a patient to interpret patient histories, examine a clinical diagnosis, or restate the recommended treatment method and follow-up to aid comprehension. The only way a medical practitioner encourages patients to ask questions or respond to queries is by directing them to input text with the help of MT. Finally, the chapter discusses contemporary techniques with Neural Machine Translation (NMT), namely multilingual, transfer learning on clinical data.

Machine translation, translates sentences from source language text to target language. NMT (Neural Machine Translation) is a machine translation mechanism based on deep learning. It has improved the weakness of the rule-based and statistical machine translation. In this paper, NMT is applied on parallel corpus pair English – Marathi. We’re proposing an NMT with a local attention mechanism. It is efficient and overcomes the OOV (Out Of Vocabulary) problem. The model performance is evaluated using the BLEU score. The results from the experiment confirm that the proposed model has outperformed the global attention mechanism and also Google translator.

This article highlights the influence of artificial intelligence (AI) on automating the translation of technical terminology in project management. In particular, the article focuses on the use of machine translation (MT), which, thanks to their ability to process large amounts of data, cloud computing, and advanced algorithms, increase the accuracy and speed of translation of technical documentation. It is noted that AI helps to reduce translation costs and improves the consistency of terminology, which is critical for the successful completion of projects with tight deadlines. Nevertheless, challenges in this field are also identified, including the high error rate in translating highly specialized terms and the ongoing need to enhance systems to meet the varied demands of users. Different translation approaches are discussed, including neural machine translation (NMT), statistical machine translation (SMT) and rule-based machine translation (RBMT), which ensure high accuracy and smoothness of translation. Integrating AI into project management systems can also optimise communication in multilingual teams. The article highlights the growing role of AI in the translation of technical terms, which has great potential to improve efficiency in project management.

Deep neural networks (DNNs) have provably enhanced the state-of-the-art natural language process (NLP) with their capability of feature learning and representation. As one of the more challenging NLP tasks, neural machine translation (NMT) becomes a new approach to machine translation and generates much more fluent results compared to statistical machine translation (SMT). However, SMT is usually better than NMT in translation adequacy and word coverage. It is therefore a promising direction to combine the advantages of both NMT and SMT. In this article, we propose a deep neural network--based system combination framework leveraging both minimum Bayes-risk decoding and multi-source NMT, which take as input the N-best outputs of NMT and SMT systems and produce the final translation. In particular, we apply the proposed model to both RNN and self-attention networks with different segmentation granularity. We verify our approach empirically through a series of experiments on resource-rich Chinese⇒English and low-resource English⇒Vietnamese translation tasks. Experimental results demonstrate the effectiveness and universality of our proposed approach, which significantly outperforms the conventional system combination methods and the best individual system output.

Over the past ten years, neural machine translation (NMT) has seen tremendous growth and is now entering a phase of maturity. Despite being the most popular solution for machine translation (MT), it performs sub-optimally on under-resourced language pairs due to lack of parallel corpora as compared to high-resourced language pairs. The implementation of NMT techniques for under-resourced language pairs is receiving the attention of researchers and has resulted in a significant amount of research for many under-resourced language pairs. In view of the growth of MT, this paper describes a set of practical approaches for investigating MT between Kannada and Tulu. These two languages belong to the family of Dravidian languages and are under-resourced due to lack of tools and resources particularly the parallel corpus for MT. Since there are no parallel corpora for the Kannada-Tulu language pair for MT, this work aims to construct a parallel corpus for this language pair. As manual construction of parallel corpus is laborious, data augmentation is introduced to enhance the size of the parallel corpus along with suitable preprocessing techniques. Different NMT schemes such as recurrent neural network (RNN) baseline, bidirectional recurrent neural network (BiRNN), transformer-based NMT with and without subword tokenization, and statistical machine translation (SMT) models are implemented for MT of Kannada-Tulu and Tulu-Kannada language pairs. Empirical results reveal that the impact of data augmentation increases the bilingual evaluation understudy (BLEU) score of the proposed models. Transformer-based models with subword tokenization outperformed the other models with BLEU scores 41.82 and 40.91 for Kannada-Tulu and Tulu-Kannada MT, respectively.

Due to its extensive use in ancient Indian religious scriptures, Sanskrit is among the oldest indigenous languages and is rightfully referred to as the language of the gods. However, it is losing favour in contemporary India. Sanskrit is not widely used in current times due in large part to the lack of resources for translation into and out of it. In recent years, machine translation (MT) has improved above and beyond the norm and is now typically performed utilising supervised learning approaches. Due to the paucity of comparable corpora for Sanskrit, new research in the unsupervised MT domain appears to have promise for Sanskrit. With the aid of manually created parallel corpora for the Sanskrit-Hindi language pair, an analysis is conducted between various modelling techniques of building a machine translation system, namely Statistical and Neural, in order to bridge the gap between Sanskrit and its contemporary successor Hindi. In order to provide a fresh viewpoint on the area as a whole, the primary benefits and drawbacks of statistical and neural machine translation has been examined in this work. Our results suggest that Neural machine translation modelling technique performs better than Statistical machine translation.

A Hybrid of Sentence-Level Approach and Fragment-Level Approach of Parallel Text Extraction from Comparable Text

Introduction to the special issue on deep learning approaches for machine translation

Corpus-based machine translation (MT) has been the main approach to developing and implementing MT systems in both academia and the industry over the last three decades. In this field, the type and size of the corpus used for training MT engines have presented problems for both statistical MT (SMT) systems as well as neural MT (NMT) systems, being the two dominant corpusbased approaches. Moreover, language pairs such as Turkish-English have been understudied within this framework. This article aims to evaluate the translation quality in Turkish-to-English custom MT systems that have been trained on different corpus sizes and types. Two NMT engines and two SMT engines were trained on the KantanMT platform using two different training corpus types with either only domain-specific cardiology corpus or this corpus plus a mixed-domain corpus. The study conducted both automatic evaluations with metrics including BLEU, F-Measure and TER, as well as a comprehensive human evaluation with metrics including fluency, A/B test, and adequacy. Lastly, the study realized a separate, subjective terminology evaluation in order to investigate how differently MT systems handle terminology, as this is a crucial aspect for specific-domain text types such as cardiology. While the automatic evaluation results suggest the SMT engines to perform better than NMT engines, all human evaluators rated the mixed-domain NMT engine as the highest performing one. However, the terminology evaluation task demonstrated SMT to still be able to perform better and to commit less terminology errors, despite the industry and academia shifting toward NMT engines.

Assembling translations from multi-engine machine translation outputs

Machine translation (MT) models usually translate a text at sentence level by considering isolated sentences, which is based on a strict assumption that the sentences in a text are independent of one another.However, the fact is that the texts at discourse level have properties going beyond individual sentences.These properties reveal texts in the frequency and distribution of words, word senses, referential forms and syntactic structures.Disregarding dependencies across sentences will harm translation quality especially in terms of coherence, cohesion, and consistency.To solve these problems, several approaches have previously been investigated for conventional statistical machine translation (SMT).With the fast growth of neural machine translation (NMT), discourse-level NMT has drawn increasing attention from researchers.In this work, we review major works on addressing discourse related problems for both SMT and NMT models with a survey of recent trends in the fields.