A Comprehensive Evaluation of Large Language Models in Mining Gene Interactions and Pathway Knowledge

Large-scale language models (LLMs) have basically changed the limits of what can be achieved by natural language processing and this was the case of GPT-3 at its very first demonstration back in 2020. This study outlines in detail their journey. It goes through all the concepts of their architecture. Besides, it describes the training process. The article records the gradual increasing of their volumetry and the consequent differences of their properties along the way. Different methods of testing these models are also indicated. Besides, it refers to their application in various fields. Besides, this research work deals with the advent of multi-modal systems capable of autonomous decision-making. We have centered our work on the landmark research that has been published in the period of 2022–2024. We reckon the self-supervised pre-training, instruction tuning, preference optimization, and reasoning-directed reinforcement learning as four main areas that contributed to the development of LLMs. These approaches have converted LLMs to be no longer just the text generators but rather the most versatile AI systems that they are. Our focus is on their use in the medical field and bioinformatics. We address the challenge of explaining the functioning of large language models (LLMs). We tell the story of in-context learning. We discuss the development of LLMs-based agents. At the end, we are identifying the major limitations of the present-time and, simultaneously, suggesting a few ideas for the future in this fast-evolving area.

In this work, we study the impact of Large-scale Language Models (LLM) on Automated Speech Recognition (ASR) of YouTube videos, which we use as a source for long-form ASR. We demonstrate up to 8% relative reduction in Word Error Eate (WER) on US English (en-us) and code-switched Indian English (en-in) long-form ASR test sets and a reduction of up to 30% relative on Salient Term Error Rate (STER) over a strong first-pass baseline that uses a maximum-entropy based language model. Improved lattice processing that results in a lattice with a proper (non-tree) digraph topology and carrying context from the 1-best hypothesis of the previous segment(s) results in significant wins in rescoring with LLMs. We also find that the gains in performance from the combination of LLMs trained on vast quantities of available data (such as C4 [1]) and conventional neural LMs is additive and significantly outperforms a strong first-pass baseline with a maximum entropy LM.

Large-scale language models, such as ChatGPT[3] and GPT-4[32], have demonstrated remarkable capabilities in generating human-like text for various applications. In this paper, we focus on two key aspects: (1) adapting open-source large language models (LLMs) for specific use cases like contract drafting using instruction tuning and parameter-efficient fine-tuning, and (2) analyzing the difference in ChatGPT’s behavior in single-role prompts compared to multi-role prompts for synthetic data generation tasks. We present a method for aligning open-source LLMs to follow instructions for customized contract drafting scenarios using parameter-efficient fine-tuning on synthetic data. Furthermore, we investigate the data quality of the synthetically generated instructions data by ChatGPT with single-role vs. multi-role prompts. Our findings reveal that the model performs better when given single-role prompts, highlighting the importance of strategically designing prompting strategy to generate better quality data using LLMs. By combining the insights from these two aspects, we explore potential implications and opportunities for enhancing generative AI solutions for practical implementations. The Contract Drafting model 1 and data 2 are released.

LLMCL-GEC: Advancing grammatical error correction with LLM-driven curriculum learning

Multimodal large-scale language modeling has become the mainstream approach in natural language processing tasks and has been applied to various cross-modal fields such as image description and visual question answering. However, large-scale language modeling has high computational complexity and a large operational scale, which presents significant challenges for deployment in many resource-constrained scenarios. To address such problems, a lightweight multimodal framework, LLaVA-GM, is proposed, based on LLaVA, which can be deployed on devices with low resource requirements and has greatly reduced model parameters. It can also be tested on common VQA tasks and achieves good performance. The main contributions and work are as follows: First, it is found that the backbone of the Vicuna language model in LLaVA is too redundant. When fine-tuning downstream tasks, a very small amount of data sets is difficult to affect the language model. It is replaced with a new Gemma language model, thereby achieving fast task-specific adaptation with fewer parameters and data. Second, in response to the problem of information redundancy, the MoE mixed expert model is introduced. This model can be used in combination with itself, combining the MoE mixed expert model with Gemma to reduce the amount of computation while maintaining performance. Directly training the entire model will lead to a decline in performance. A multi-stage training strategy is adopted to maintain performance. First, the MLP layer is trained for visual adaptation, then the entire Gemma model is trained to improve multimodal capabilities, and finally only the MoE layer is trained for sparsification to ensure a smooth transition from dense models to sparse models. The experiment was tested on multiple VQA datasets and achieved good performance, confirming the potential of this compact model in downstream multimodal applications.

This study investigates the potential of artificial intelligence, particularly Natural Language Processing and large-scale language models, to improve resource management and service access for individuals with autism in Alabama. The research aims to explore and evaluate the potential of AI-driven tools to address challenges in navigating complex datasets and supporting social work practices. We designed and tested AI systems, including general language models, domain-specific chatbots powered by advanced language models, and a Retrieval-Augmented Generation framework. A standardized set of queries was used to simulate real-world scenarios encountered by social workers engaged in autism service coordination. System performance was evaluated based on precision, recall, and response accuracy. Results demonstrated that the Retrieval-Augmented Generation (RAG) framework achieved superior performance compared to traditional NLP methods and general-purpose language models, attaining approximately 90–96% precision and recall across evaluated query types. RAG outperformed the domain-specific GPT-4 chatbot by approximately 5–12 percentage points in F1 score, with the largest gains observed for queries requiring geographic specificity, multiple constraints, or complex contextual understanding. The integration of domain-specific retrieval significantly enhanced the accuracy, contextual relevance, and usability of generated responses. The findings highlight the transformative potential of AI-driven tools in improving social work efficiency and enhancing healthcare equity. By streamlining care coordination and delivering accurate, contextually relevant information, these systems offer scalable solutions to improve access to autism-related services. Future research should focus on addressing data quality, minimizing biases, and ensuring ethical deployment to build trust and support widespread adoption of these tools.

An interesting application for social robots is to act as a presenter, for example as a museum guide. In this paper, we present a fully automated system architecture for building adaptive presentations for embodied agents. The presentation is generated from a knowledge graph, which is also used to track the grounding state of information, based on multimodal feedback from the user. We introduce a novel way to use large-scale language models (GPT-3 in our case) to lexicalise arbitrary knowledge graph triples, greatly simplifying the design of this aspect of the system. We also present an evaluation where 43 participants interacted with the system. The results show that users prefer the adaptive system and consider it more human-like and flexible than a static version of the same system, but only partial results are seen in their learning of the facts presented by the robot.

Language steganography in social networks primarily focuses on embedding secret information into social media text efficiently to achieve covert communication. The misuse of such techniques could pose significant potential threats to public cyberspace, such as the spread of malicious code, commands, or viruses. Existing social text steganalysis techniques mainly focus on the analysis of individual social media texts. However, the information content in a single text is very limited, leading to poor detection performance in practical applications. To address this challenge, this paper proposes a social text steganalysis method that combines large-scale language models with common-sense knowledge graphs (STLC-KG). This method first uses knowledge graphs to expand the knowledge contained in the text under investigation, enriching its linguistic expression, and then utilizes large-scale language models to extract the linguistic features of the social text. The results of tests conducted on three mainstream social media platforms demonstrate that the proposed method significantly improves the performance of social text steganalysis.

In this paper, we introduce ACAT-G, an interactive dialogue learning framework that incorporates constant human feedback into fine-tuning language models in order to assist conditioned dialog generation. The system takes in a limited amount of input from a human and generates personalized response corresponding to the context of the conversation within natural dialog time-frame. By combining inspirations from online learning, reinforcement learning, and large scale language models, we expect this project to provide a foundation for human-in-the-loop conditional dialog generation tasks.

Current approaches to testing and debugging NLP models rely on highly variable human creativity and extensive labor, or only work for a very restrictive class of bugs. We present AdaTest, a process which uses large scale language models (LMs) in partnership with human feedback to automatically write unit tests highlighting bugs in a target model. Such bugs are then addressed through an iterative text-fix- retest loop, inspired by traditional software development. In experiments with expert and non-expert users and commercial / research models for 8 different tasks, AdaTest makes users 5-10x more effective at finding bugs than current approaches, and helps users effectively fix bugs without adding new bugs.

Transcription factors (TFs) and microRNAs (miRNAs) are two major types of regulators of gene expression, at transcriptional and post-transcriptional levels, respectively. By gathering their gene regulatory relationships, gene regulatory networks (GRNs) could be formed. A network motif is a type of connection pattern among a set of nodes which appears significantly more frequently than in random networks. Investigations of the network motifs often yield biological insights into the nature of the network. The previous study on miRNA-TF regulation networks concentrated on animals, and relied heavily on computational predictions. The authors collected data concerning miRNA regulation and transcriptional regulation relationships in Arabidopsis from publicly available databases, and further incorporated them with the protein-protein interaction data. All the data in the author's collection are supported by experiments. They screened the network motifs, whose size ranges between 1 and 4. The biological implications of the motifs were further analysed, and a flower development related network was constructed as an example. In this example, they illustrated the relevance of the network with the given process, and proposed the association of several genes with flowers by a network cluster identification. In this study, they analysed the properties of the GRN in Arabidopsis, and discussed their biological implications, as well as their potential applications.

The cinematographic vanguard has historically been important in fostering cinematographic creation and innovation. In the digital age, its study acquires a new dimension of relevance, since the analysis of these vanguard movements provides students with tools to understand how cinema has evolved as a means of artistic expression and social manifestation. This knowledge is essential to developing robust critical thinking, capable of evaluating not only the content of a film, but also its form, its historical context, and its cultural relevance from and in its immediate environment. With the arrival of large-scale language models (LLM) and generative artificial intelligence (IAGen), students face an increasingly complex social landscape where technologies are transforming the production, distribution and analysis of movies. LLMs can generate scripts or critical analyses, while IAGens can create visual effects or even entire film sequences, modifying the cinematic concept. This essay analyzes the importance of cinematic avant-garde in the development of students’ critical thinking, especially in the current context of the proliferation of large-scale language models in the form of generative artificial intelligences.

Large-scale language models such as DNABert and LOGO aim to learn optimal gene representations and are trained on the entire Human Reference Genome. However, standard tokenization schemes involve a simple sliding window of tokens like k-mers that do not leverage any gene-based semantics and thus may lead to (trivial) masking of easily predictable sequences, and subsequently inefficient Masked Language Modeling (MLM) training. Therefore, we propose a novel masking algorithm, GENEMASK, for MLM training of gene sequences, where we randomly identify positions in a gene sequence as mask centers and locally select the span around the mask center with the highest Normalized Pointwise Mutual Information (NPMI) to mask. We observe that in the absence of human-understandable semantics in the genomics domain (in contrast, semantic units like words and phrases are inherently available in NLP), GENEMASK-based models substantially outperform the SOTA models (DNABert and LOGO) over four benchmark gene sequence classification datasets in five few-shot settings (10 to 1000-shot). More significantly, the GENEMASK-based DNABert model is trained for less than one-tenth of the number of epochs of the original SOTA model. We also observe a strong correlation between top-ranked PMI tokens and conserved DNA sequence motifs, which may indicate the incorporation of latent genomic information. The codes (including trained models) and datasets are made publicly available at https://github.com/roysoumya/GeneMask.

A wide variety of natural language tasks are currently being addressed with large-scale language models (LLMs).These models are usually trained with a very large amount of unsupervised text data and adapted to perform a downstream natural language task using methods like fine-tuning, calibration or in-context learning.In this work, we propose an approach to adapt the prior class distribution to perform text classification tasks without the need for labelled samples and only a few in-domain sample queries.The proposed approach treats the LLM as a black box, adding a stage where the model posteriors are calibrated to the task.Results show that these methods outperform the un-adapted model for different number of training shots in the prompt and a previous approach where calibration is performed without using any adaptation data.

This paper presents a system of large-scale language model adaptation for daily generated big-size text corpus using MapReduce in a cloud environment. Our large-scale trigram language model, consis...