Knowledge Transfer for Utterance Classification in Low-Resource Languages

In this paper, we propose a new model for text line segmentation and classification, which consists of convolutional and two-layer bi-directional long short-term memory (BiLSTM) networks. Trained on the synthetic text dataset, it performs excellently when predicting the real data. Without labelling every line on the real data, a generalized standard for evaluating the accuracy is proposed. We also propose a simplified IoU loss to improve the execution speed greatly. In the experiments, it achieves 98.1% line segmentation accuracy and 99.5% classification accuracy on the English fiction Pride and Prejudice by Jane Austen, and achieves 98.5% line segmentation accuracy and 99.7% classification accuracy on the The Secret Of Plato's Atlantis by John Arundell, outperforming the traditional methods. Furthermore, for 1024 × 724 input samples, it gets 2.95 FPS speed when using a Tesla K80 GPU. Index Terms—Text line segmentation, Text classification, Synthetic text, BiLSTM, Convolutional network.

Over the past few years, Large Language Models of Code (Code LLMs) have started to have a significant impact on programming practice. Code LLMs are also emerging as building blocks for research in programming languages and software engineering. However, the quality of code produced by a Code LLM varies significantly by programming language. Code LLMs produce impressive results on high-resource programming languages that are well represented in their training data (e.g., Java, Python, or JavaScript), but struggle with low-resource languages that have limited training data available (e.g., OCaml, Racket, and several others). This paper presents an effective approach for boosting the performance of Code LLMs on low-resource languages using semi-synthetic data. Our approach, called M ulti PL-T, generates high-quality datasets for low-resource languages, which can then be used to fine-tune any pretrained Code LLM. M ulti PL-T translates training data from high-resource languages into training data for low-resource languages in the following way. 1) We use a Code LLM to synthesize unit tests for commented code from a high-resource source language, filtering out faulty tests and code with low test coverage. 2) We use a Code LLM to translate the code from the high-resource source language to a target low-resource language. This gives us a corpus of candidate training data in the target language, but many of these translations are wrong. 3) We use a lightweight compiler to compile the test cases generated in (1) from the source language to the target language, which allows us to filter our obviously wrong translations. The result is a training corpus in the target low-resource language where all items have been validated with test cases. We apply this approach to generate tens of thousands of new, validated training items for five low-resource languages: Julia, Lua, OCaml, R, and Racket, using Python as the source high-resource language. Furthermore, we use an open Code LLM (StarCoderBase) with open training data (The Stack), which allows us to decontaminate benchmarks, train models without violating licenses, and run experiments that could not otherwise be done. Using datasets generated with M ulti PL-T, we present fine-tuned versions of StarCoderBase and Code Llama for Julia, Lua, OCaml, R, and Racket that outperform other fine-tunes of these base models on the natural language to code task. We also present Racket fine-tunes for two very recent models, DeepSeek Coder and StarCoder2, to show that M ulti PL-T continues to outperform other fine-tuning approaches for low-resource languages. The M ulti PL-T approach is easy to apply to new languages, and is significantly more efficient and effective than alternatives such as training longer.

In this article, the method of text classification using a convolutional neural network is presented. The problem of text classification is formulated, the architecture and the parameters of a convolutional neural network for solving the problem are described, the steps of the solution and the results of classification are given. The convolutional network which was used was trained to classify the texts of the news messages of Internet information portals. The semantic preprocessing of the text and the translation of words into attribute vectors are generated using the open word2vec model. The analysis of the dependence of the classification quality on the parameters of the neural network is presented. The using of the network allowed obtaining a classification accuracy of about 84%. In the estimation of the accuracy of the classification, the texts were checked to belong to the group of semantically similar classes. This approach allowed analyzing news messages in cases where the text themes and the number of classification classes in the training and control samples do not equal.

Convolutional neural networks (CNNs) have become a useful tool for a wide range of applications such as text classification. However, CNNs are not always sufficiently accurate to be useful in certain applications. The selection of activation functions within CNN architecture can affect the efficacy of the CNN. However, there is limited research regarding which activation functions are best for CNN text classification. This study tested sixteen activation functions across three text classification datasets and six CNN structures, to determine the effects of activation function on accuracy, iterations to convergence, and Positive Confidence Difference (PCD). PCD is a novel metric introduced to compare how activation functions affected a network’s classification confidence. Tables were presented to compare the performance of the activation functions across the different CNN architectures and datasets. Top performing activation functions across the different tests included the symmetrical multi-state activation function, sigmoid, penalised hyperbolic tangent, and generalised swish. An activation function’s PCD was the most consistent evaluation metric during activation function assessment, implying a close relationship between activation functions and network confidence that has yet to be explored.

Due to the continuous growth and advancement of technology, an enormous volume of text data is generated daily across various sources including social media platforms, websites, search engines, healthcare records, and news articles. Extracting meaningful patterns from text data, such as viewpoints, related theories, journal distribution, facts, and the development of online news text, is a challenging task due to the varying lengths of the texts. One issue arises from the length of the text data itself, and another challenge lies in extracting valuable features, especially in news articles. In the deep learning models, the convolutional neural networks (CNNs) are capable of capturing local features in text data, but unable to capture the structural information or semantic relationships between words. Consequently, a sole CNN network often yields poor performance in text classification tasks, whereas the Gated Recurrent Unit (GRU) is adept at effectively extracting semantic information and understanding the global structural relationships present in textual data. This paper presents a solution to the problem by introducing a new text classification that integrates the strengths of CNN and GRU. The proposed hybrid models incorporate word vectorization and word dispersion in parallel. Initially, the model trains word vectors using the Word2vec model and then leverages the GRU model to capture semantic information from text sentences. Subsequently, the CNN method is employed to capture crucial semantic features, leading to classification using the SoftMax layer. Experimental findings demonstrated that the proposed hybrid GRU_CNN model outperformed and achieved accuracy 97.73% as compared to individual CNN, LSTM, and GRU models in terms of classification effectiveness and accuracy.

Objects and processes classification is a common experimental problem. Its solution, first of all, is needed in automatic diagnosis systems, for example, to determine the equipment operation state through the diagnostic signal or to identify abnormalities with medical images. With the development of convolutional neural networks, new prospects for solving such problems have risen up. However, the classification accuracy that can be achieved on these networks is not sufficient enough for all diagnosis issues. It is subject to, for example, timely diagnosis of the onset of transient phenomena. At the same time, another type of neural network, Kohonen self-organizing maps, has a conceptual property for training on unclassified set of classes, that is, giving the opportunity to solve such an issue. Therefore, the accuracy enhancement of the classification on the basis of Kohonen networks implementation in the architecture of convolutional networks is a relevant objective and has practical significance. The article analyzes the means of improving the accuracy of convolutional neural networks and arising problems solution. The ways of increasing the proportion of correct clustering on Kohonen networks due to the ‘growth’ of its grid shape in determining new classes in the learning process are also given. It is this property that makes it possible to recognize transient phenomena. It is determined that the existing solutions of the combination of Kohonen networks and convolutional networks are aimed at improving the efficiency of only self-organizing maps, with the purpose to improve the accuracy of classification by convolutional networks, it became necessary to develop a new architecture. The paper provides a description of this issue. Since the initial information of the Kohonen networks is the weighting matrix values of the grid shape neurons, it was necessary to associate it with the representation of the images in order to process the diagnostic images. The paper proposes the concept of a built-in associative array block based on Kohonen networks. According to the proposed method, a software implementation of a hybrid neural network is developed. The formulation and results of computational experiments are presented. The efficiency of the proposed method is experimentally proved. Keywords: convolutional neural networks, CNN, Kohonen self-organizing maps, SOM, signal classification, image classification.

Text classification is a classic task in the field of natural language processing. however, the existing methods of text classification tasks still need to be improved because of the complex abstraction of text semantic information and the strong relecvance of context. In this paper, we combine the advantages of two traditional neural network model, Long Short-Term Memory(LSTM) and Convolutional Neural Network(CNN). LSTM can effectively preserve the characteristics of historical information in long text sequences, and extract local features of text by using the structure of CNN. We proposes a hybrid model of LSTM and CNN, construct CNN model on the top of LSTM, the text feature vector output from LSTM is further extracted by CNN structure. The performance of the hybrid model is compared with that of other models in the experiment. The experimental results show that the hybrid model can effectively improve the accuracy of text classification.

Text classification is one of the important applications of Natural Language Processing (NLP) in which text is classified into pre-defined categories or classes. Several state-of-the-arts have already been conducted on text classification in different Indian languages, however, limited researches on text classification performed in Odia. Moreover, the news headlines classification is a text classification technique which is performed in our study mainly focusing on Odia language. The Convolutional Neural Network (CNN) based Deep Learning technique is used to classify the Odia news headlines dataset available at Kaggle repository. The dataset consisting of approx. 15,200 Odia news headlines is split into training and testing sets of 80:20 ratio and is categorized into different classes, such as sports, business, and entertainment by using CNN. The efficacy and efficiency of the proposed CNN-based Odia news headlines classification is evaluated in contrary to other existing machine learning and deep learning models, such as Neural Networks, LSTM (Long Short-Term Memory), and Bi-LSTM (Bidirectional Long Short-Term Memory) with respect to different performance metrics like precision, accuracy, recall, F1-score, training time, and AUC-ROC (Area Under the Receiver Operating Characteristic Curve). The simulation results revealed that the proposed CNN model outperformed with respect to other models.

Text representations in text classification usually have high dimensionality and are lack of semantics, resulting in poor classification effect. In this paper, TF-IDF is optimized by using optimization factors, then word2vec with semantic information is weighted, and the single-text representation model CD_STR is obtained. Based on the CD_STR model, the latent semantic index (LSI) and the TF-IDF weighted vector space model (T_VSM) are merged to obtain a fusion model, CD_MTR, which is more efficient. The text classification method MTR_MCNN of the fusion model CD_MTR combined with convolutional neural network is further proposed. This method first designs convolution kernels of different sizes and numbers, allowing them to extract text features from different aspects. Then the text vectors trained by the CD_MTR model are used as the input to the improved convolutional neural network. Tests on two datasets have verified that the performance of the two models, CD_STR and CD_MTR, is superior to other comparable textual representation models. The classification effect of MTR_MCNN method is better than that of other comparison methods, and the classification accuracy is higher than that of CD_MTR model.

Multi-Language Speech-to-Text Translation (ST) plays a pivotal role in bridging linguistic barriers by converting spoken language into written text across different languages.This project aims to develop a robust ST model tailored for low-resource Indian languages, specifically targeting the Indo-Aryan and Dravidian language families.The dataset used consists of speeches from conferences and TED Talks, along with their corresponding transcriptions in English (source language) and translations in Hindi, Bengali, and Tamil (target languages).By tackling the lack of data and disparities in attention within low-resource languages, the paper strives to create an efficient ST system capable of real-world deployment.Additionally, existing resources in related languages are leveraged and word-level translation resources are explored to enhance translation accuracy.

This study evaluates the dual role of translators and interpreters working for NGOs involved in the health sector in developing countries. Given the crucial importance of public health in a nation’s development, we assess the way grassroots participation in healthcare projects and activities is enhanced through the translation of healthcare information into low-resource languages. This study, epistemologically grounded in the theory of planned behaviour (TPB), adopts a qualitative and quantitative methodological approach to correlate the TPB and the target-language choices of translator-interpreters and the translation strategies they adopt working with low-resource languages. The survey responses and audio recordings analysed are those of translators or interpreters working in a rural context in the Anglophone regions of Cameroon. The target language in question is Pidgin English, a low-resource oral language. The evidence gathered leads to the finding that various de-terminologization strategies are employed to ensure that messages are adequately disseminated and that the local community members are involved in healthcare activities. The study seeks to bring together translation, interpreting, the TPB, and international development in which the involvement of NGOs underscores North–South cooperation in the strategic domain of global healthcare.

Classification of Bengali accident texts describing daily life incidents such as injuries, deaths, and various damages is essential for safety management systems, but deep learning approaches for this low-resource language are still limited. This study examines multiple deep learning models for both binary and multi-class classification of Bengali accident texts. We develop a voting ensemble method designed for Bengali language characteristics, combining traditional deep learning models with hybrid architectures to tackle low-resource language challenges. Experiments on Bengali accident datasets achieve strong results: 95.9% accuracy for binary classification and 94.6% accuracy for multi-class classification across ten accident categories. Our ensemble approach outperforms baseline models by 1.8% and 9.5%, establishing new benchmarks for Bengali accident text classification. We also implement LIME (Local Interpretable Model-agnostic Explanations) to provide clear explanations for safety-critical decisions. These results demonstrate the effectiveness of ensemble deep learning for safety-related text classification in low-resource language settings.

Sentiment text classification is a natural language processing technique for recognizing and extracting subjective information in text, such as emotions, attitudes, and opinions. Traditional models such as BERT and XLNet have achieved brilliant results in text classification problems. With the rise of large language model technology, using generative large models to improve text classification accuracy has become a new research direction. However, compared with traditional classification models, large language models have a slight disadvantage in language classification tasks. In this paper, we propose a prompt-based approach to enhance the accuracy of large language models for text classification using image prompt information on multimodal datasets. First, we illustrate the principle of consistency between image and textual information. Second, we propose a multimodal framework for sentiment analysis of images and text, which realizes the prediction of sentiment tendency for both image and textual data by injecting the image prompt information into the text and into the Large Language Model. Finally, we designed experiments and evaluated them using real multimodal datasets to verify the effectiveness and accuracy of the framework.

Land-use information provides a direct representation of the effect of human activities on the environment, and an accurate and efficient land-use classification of remote sensing images is an important element of land-use and land-cover change research. To solve the problems associated with traditional land-use classification methods (e.g., rapid increase in dimensionality of data, inadequate feature extraction, and low running efficiency), a method that combines object-oriented approach with deep convolutional neural network (COCNN) is presented. First, a multi-scale segmentation algorithm is used to segment images to generate image segmentation regions with high homogeneity. Second, a typical rule set of feature objects is constructed on the basis of the object-oriented segmentation results, and the segmentation objects are classified and extracted to form a training sample set. Third, a convolutional neural network (CNN) model structure is modified to improve classification performance, and the training algorithm is optimized to avoid the overfitting phenomenon that occurs during training using small datasets. Ten land-use types are classified by using the remote sensing images covering the area around Fuxian Lake as an example. By comparing the COCNN method with the method based solely on CNN, precision and kappa index were selected to evaluate the classification accuracy of the two methods. For the COCNN method, on the basis of the classification statistics, precision and kappa index coefficients are 96.2% and 0.96, respectively, which are 8.98% and 0.1 higher than those of the method based solely on CNN. Experimental results show that the COCNN method reasonably and efficiently combines object-oriented and deep learning approaches, thereby effectively solving the problem of the inaccurate classification of typical features with better classification accuracy than the simple use of CNN.

Convolutional neural network (CNN) and recurrent neural network (RNN) are two popular architectures used in text classification. Traditional methods to combine the strengths of the two networks rely on streamlining them or concatenating features extracted from them. In this paper, we propose a novel method to keep the strengths of the two networks to a great extent. In the proposed model, a convolutional neural network is applied to learn a 2D weight matrix where each row reflects the importance of each word from different aspects. Meanwhile, we use a bidirectional RNN to process each word and employ a neural tensor layer that fuses forward and backward hidden states to get word representations. In the end, the weight matrix and word representations are combined to obtain the representation in a 2D matrix form for the text. We carry out experiments on a number of datasets for text classification. The experimental results confirm the effectiveness of the proposed method.