A Scalable Retrieval-Augmented Generation Pipeline for Domain-Specific Knowledge Applications

This paper presents a system design for automating incident update responses in telecommunication service providers using Large Language Models (LLMs). We address key challenges in current incident management practices: tailoring customer-faced communications while also maintaining cost efficiency and user experience. The proposed system design implements a caching mechanism to reduce operational costs and response latency. Rather than using semantic similarity search, our system employs a strict key-match caching approach that ensures consistent responses while minimizing computational overhead. We selected DeepSeek-V3 API as our primary LLM based on comprehensive cost-performance analysis. Our prompt engineering approach incorporates persona definition, chain-of-thought reasoning, and constitutional AI principles to generate tailored responses. Experimental results demonstrate significant efficiency gains, validated through quantitative analysis. Eliminating redundant LLM calls avoids linear cost growth, while reduced latency improves speed by 99%. Additionally, LLM response quality successfully passed User Acceptance Test. This work contributes to the practical implementation of LLMs in Customer Relationship Management systems by balancing cost efficiency, response quality, and customer experience.

The subject of the study is the analysis and improvement of methods for determining the relevance of project names to the information content of purchases using large language models. The object of the study is a database containing the names of projects and purchases in the field of electric power industry, collected from open sources. The author examines in detail such aspects of the topic as the use of TF-IDF and cosine similarity metrics for primary data filtering, and also describes in detail the integration and evaluation of the effectiveness of large language models such as GigaChat, GPT-3.5, and GPT-4 in text data matching tasks. Special attention is paid to the methods of clarifying the similarity of names based on reflection introduced into the prompta of large language models, which makes it possible to increase the accuracy of data comparison. The study uses TF-IDF and cosine similarity methods for primary data analysis, as well as large GigaChat, GPT-3.5 and GPT-4 language models for detailed verification of the relevance of project names and purchases, including reflection in model prompta to improve the accuracy of results. The novelty of the research lies in the development of a combined approach to determining the relevance of project names and purchases, combining traditional methods of processing text information (TF-IDF, cosine similarity) with the capabilities of large language models. A special contribution of the author to the research of the topic is the proposed methodology for improving the accuracy of data comparison by clarifying the results of primary selection using GPT-3.5 and GPT-4 models with optimized prompta, including reflection. The main conclusions of the study are confirmation of the prospects of using the developed approach in the tasks of information support for procurement processes and project implementation, as well as the possibility of using the results obtained for the development of text data mining systems in various sectors of the economy. The study showed that the use of language models makes it possible to improve the value of the F2 measure to 0.65, which indicates a significant improvement in the quality of data comparison compared with basic methods.

DocuMind AI is an advanced document-based question answering system developed using Retrieval-Augmented Generation (RAG) to address challenges in extracting meaningful information from unstructured data. Organizations often struggle to extract useful information from large volumes of unstructured documents. Traditional keyword-based search methods often fail to capture semantic context, resulting in irrelevant outputs. To overcome this limitation, the proposed system integrates semantic search with large language models (LLMs) to generate accurate and context-aware responses. The system utilizes vector embeddings to represent document content and retrieve relevant information efficiently. It is implemented using a Java Spring Boot backend integrated with LangChain4j, which manages document ingestion, text chunking, embedding generation, and retrieval processes. Users interact with the system through a React.js-based interface, allowing them to upload documents and query in natural language. The system supports multiple document formats such as PDF and DOCX, ensuring flexibility in real-world applications. Additionally, JWTbased authentication is implemented to provide secure access to the system. The modular and scalable architecture makes DocuMind suitable for enterprise-level deployment, bridging the gap between traditional information retrieval systems and modern AI-driven solutions. Keywords: Retrieval-Augmented Generation, Natural Language Processing, Large Language Models, Vector Database, Java Spring Boot, LangChain4j, PostgreSQL pgvector, Cosine Similarity.

Reverse engineering is a cybersecurity process that focuses on understanding the underlying functionality of software or malware. This is an arduous process that demands large amounts of time and effort from cybersecurity practitioners. Large Language Models (LLMs) offer a potential solution to this problem. LLMs have worked their way into various fields of cybersecurity in recent years, including incident response and malware classification. However, LLMs have historically struggled with low-level code comprehension: a necessary part of reverse engineering. While LLMs can generate code and explain its function on the surface level, they struggle to grasp the wider context. In this paper, we utilize parameter-efficient fine-tuning to train LLMs to generate contextual comments for x86 assembly code in an effort to expedite reverse engineering. We choose LLMs from several parameter classes in each of the Qwen2.5-Coder, CodeLlama, and CodeGemma families to fine-tune on a dataset of x86 assembly code. We evaluate each model's performance on cross entropy loss and cosine similarity before and after fine-tuning. We observe promising results, with a significant boost in similarity score for five out of the seven LLMs selected. This is particularly evident in the 0.11 increase in simialrity score for Qwne2.5-Coder-7B and the 0.18 increase in similarity score for CodeLlama-7B after fine-tuning.

Benchmark evaluation of large language models in acute myeloid leukemia prognosis

Large Language Models (LLMs) are state-of-the-art in Natural Language Processing (NLP) tasks but are extremely challenging to fine-tune with their size and computational needs. The current research targets the fine-tuning of the OpenLLaMA-3B model for personality development tasks with parameter-efficient fine-tuning (PEFT) via LoRA and QLoRA. To deal with hardware limitations, 4-bit quantization through BitsAndBytes was used, lowering memory without affecting accuracy. A FAISS-based Retrieval-Augmented Generation (RAG) pipeline was also used to improve contextual reliability. Semantic similarity (cosine similarity), BLEURT, ROUGE, and human evaluation were used to evaluate the model. The results of this experiment show that semantic similarity greatly increasing from 0% before fine-tuning to 80% after fine-tuning, demonstrating the benefit of PEFT and quantization in domain adaptation. The work illustrates that by leveraging effective fine-tuning, quantization, and retrieval augmentation, LLMs can be deployed at scale under limited resources while providing high contextual accuracy for personality development purposes. This approach not only enhances knowledge of the job, but it also has practical scalability for educational and self-improvement purposes. The findings highlight a viable path forward for applying small, yet strong, LLMs to personalized learning and adaptive human-AI interaction.

Retrieval augmented generation (RAG) is a popular framework for question answering that is powered by two large language models (LLMs): an embedding model that retrieves context documents from a database that are relevant to a given question, and a generator model that uses the retrieved context to generate an answer to the question.Both the embedding and generator models can be fine-tuned to increase performance of a RAG pipeline on a new task, but multiple fine-tuning strategies exist with different costs and benefits.In this paper, we evaluate and compare several RAG fine-tuning strategies, including independent, joint, and two-phase fine-tuning.In our experiments, we observe that all of these strategies achieve about equal improvement in EM and F1 generation quality metrics, although they have significantly different computational costs.We conclude the optimal fine-tuning strategy to use depends on whether the training dataset includes context labels and whether a grid search over the learning rates for the embedding and generator models is required.

Introduction Large language models (LLMs) have the potential to realize accurate risk stratification and disease prediction by integrating multimodal data, such as electronic health records, medical images, and genomic profiles. However, in complex tasks like atherosclerosis risk estimation, the design of the prompt is critical to bring out the performance of LLMs. Several researches have demonstrated that in-context learning, where the prompt contains some cases, improves the performance of LLMs when the contained cases are carefully selected. Because manual selection usually requires enormous costs, automatic case selection has been desired. Methods This study aims to improve the effectiveness of in-context learning in LLM for atherosclerosis risk prediction by introducing a strategy for case selection. Our method involves two key components: feature selection and case selection. In the feature selection phase, we compute the mutual information between the clinical features and atherosclerosis risk using the database to extract the essential clinical features. This feature selection removes the unimportant features that disturb the performance of the case selection mechanism. Then, in the case selection phase, we select several cases from the database that are the most similar to the patient being diagnosed. This study compares three metrics to compute the similarity score: - Mahalanobis distance (trained via large margin nearest neighbor classification; LMNN [1]) - Cosine similarity (between the vector representations in KNN-augmented in-context example selection; KATE [2]) - Euclidean distance (between the prompt features) This combined strategy supplies the LLM with informative cases and thereby enhances the diagnostic performance. Results We used a fine-tuned Llama3-8B model for our target language and analyzed a dataset of 117,709 cases. Each case contained 96 features that were collected from annual health check-up data. From this dataset, 1,000 cases each were randomly selected for validation and test datasets. The remaining data were used as the database in our system. We considered a binary classification task and assigned the positive label when the Cardio-Ankle Vascular Index (CAVI) was greater than 8.0. The proportion of selected features was adjusted in 10% increments to maximize the F1 score on the validation dataset. The result shows that our proposed method, the strategic feature and case selections, significantly improved the F1 score for the test dataset by around 2% compared with zero-shot prompt and random selection (see the attached table). Among three similarity metrics, Euclidean distance yielded the highest F1 scores. We also observed that the feature selection phase with the adjusted number of selected features improved the F1 score in almost all settings. Conclusion Our strategy for feature and case selections improved the performance of in-context learning for LLM-based atherosclerosis risk prediction.

Research Context: Nearest Neighbor Search (NNS) is a fundamental problem in computing, with applications in recommendation systems, semantic search, information retrieval, and Retrieval-Augmented Generation (RAG) in Large Language Models (LLMs). High-dimensional datasets with millions of vectors make exact search computationally intractable, demanding efficient approximations. Practical Problem: Approximate Nearest Neighbor Search (ANNS) has become the standard approach, with graph-based methods standing out. However, structures such as Hierarchical Navigable Small World (HNSW) graphs impose high construction and update costs, which limits their scalability in real-world scenarios. Proposed Solution: This paper proposes the integration of Canopy Clustering as a divide-and-conquer heuristic to mitigate the computational burden of HNSW graph construction under cosine similarity, enabling more cost-effective ANNS pipelines in information retrieval systems. Related IS Theory: The research aligns with Information Systems theories on efficiency of knowledge retrieval, and the optimization of data-intensive processes in decision-support environments. It builds upon principles of data organization and access structures in IS, particularly the trade-offs between accuracy and computational feasibility in large-scale information retrieval. Research Method: Controlled experiments were conducted to evaluate the applicability of divide-and-conquer and Canopy Clustering prior to HNSW construction. Comparative analyses measured build time, query latency, and recall performance. Summary of Results: Preliminary findings suggest that Canopy Clustering can reduce construction costs while sustaining recall levels adequate for IS applications. Contributions and Impact to IS area: The work introduces a novel heuristic combination to enhance the scalability of graph-based similarity search indexing, providing methodological advances that may extend to other high-dimensional data processing tasks. It offers actionable insights for designing more efficient and viable intelligent information systems.

Patient experiences and perspectives are essential for shaping patient-centered healthcare. While large language models (LLMs) in healthcare are typically applied to specific clinical or patient-facing tasks, they have not been used for qualitative patient preference assessment, which often relies on thematic analysis to understand patient views expressed in interviews or focus groups. LLMs show initial promise for performing inductive thematic analysis of healthcare interview or focus group transcripts, yet no empirical studies have investigated LLMs to facilitate qualitative patient preference assessment. We employed the open-source Hermes-3-Llama-3.1-70B LLM to perform inductive thematic analysis on focus group transcripts from a previously published qualitative patient preference assessment study using three optimized prompt frameworks, and evaluated semantic similarity of LLM generated themes against human-analyzed themes using the Sentence-T5-XXL language embedding model. Sentence-level theme similarity was assessed using Jaccard similarity coefficients (0-1 range), computing coefficient scores across a broad range of discrete cosine similarity thresholds. We further evaluated LLM themes for similarity in lexical diversity and reading grade-level metrics and benchmarked semantic similarity results with published similarity thresholds previously used with qualitative healthcare data. All prompt frameworks generated themes with median Jaccard similarity coefficients with human-analyzed themes between 0.46-0.64, indicating moderate semantic overlap. Our best-performing framework instructed to pursue thematic saturation scored closest to human-analyzed themes on all reading grade-level metrics, and demonstrated 12% higher semantic overlap with human-analyzed themes compared to published benchmarks. Our worst-performing framework produced themes with moderate semantic overlap and hallucinated findings unidentified in human-analyzed themes. We demonstrate that LLMs can perform inductive thematic analysis of qualitative patient preference data, producing themes substantively similar in content and style to human-analyzed themes when augmented with sufficient domain-specific context. While LLMs may augment thematic analysis, the contextual nature of qualitative analysis remains a challenge requiring collaborative LLM frameworks integrating human expertise.

The application of deep learning to tabular data remains an ongoing challenge, with tree-based models such as XGBoost consistently outperforming neural network methods in most real-world scenarios. Recent advances in Large Language Models (LLMs) highlight their ability to generalize to new tasks via few-shot or one-shot prompting, yet questions remain about their utility in structured, tabular domains—particularly for resource-efficient Small Language Models (SLMs). In this work, we systematically evaluate the effectiveness of both large and small transformer-based language models for tabular classification tasks using few-shot strategies. Our approach investigates input serialization schemes and prompt engineering to maximize performance in low-data regimes. Experiments on benchmark datasets—including Diabetes, Heart Failure, and German Credit Risk—demonstrate that well-designed SLMs can approach, and occasionally match, the performance of much larger models, substantially reducing the need for extensive labeled data and retraining. However, further advances are required for language models to consistently rival the strongest tree-ensemble baselines. Our findings support the idea that SLMs, when properly prompted, offer a promising, flexible, and label-efficient alternative for automating and dynamizing machine learning pipelines on tabular data.

The rapid advancement of large language models (LLMs) has intensified the need for effective mechanisms to transform continuous multimodal data into discrete representations suitable for language-based processing. Discrete tokenization, with vector quantization (VQ) as a central approach, offers both computational efficiency and compatibility with LLM architectures. Despite its growing importance, there is a lack of a comprehensive survey that systematically examines VQ techniques in the context of LLM-based systems. This work fills this gap by presenting the first structured taxonomy and analysis of discrete tokenization methods designed for LLMs. We categorize 8 representative VQ variants that span classical and modern paradigms and analyze their algorithmic principles, training dynamics, and integration challenges with LLM pipelines. Beyond algorithm-level investigation, we discuss existing research in terms of classical applications without LLMs, LLM-based single-modality systems, and LLM-based multimodal systems, highlighting how quantization strategies influence alignment, reasoning, and generation performance. In addition, we identify key challenges including codebook collapse, unstable gradient estimation, and modality-specific encoding constraints. Finally, we discuss emerging research directions such as dynamic and task-adaptive quantization, unified tokenization frameworks, and biologically inspired codebook learning. This survey bridges the gap between traditional vector quantization and modern LLM applications, serving as a foundational reference for the development of efficient and generalizable multimodal systems. A continuously updated version is available at: https://github.com/jindongli-Ai/LLM-Discrete-Tokenization-Survey.

This study evaluates the effect of retrieval embeddings and similarity metrics at the generative component (LLaMA3). No fine-tuning or decoding variation was applied to the generator, ensuring that any performance differences were due to retrieval quality alone. This design enables a focused evaluation of how retrieval configurations influence the final output in a controlled RAG setup, particularly for primary school topics in sports and health (PJOK). We point out that RAG is a technique that helps large language models make fewer mistakes by using outside information, which improves the accuracy of facts and the relevance of the context. The research compares various embedding models and retrieval similarity metrics to evaluate their effectiveness in retrieving relevant information. The baai/bge-m3 and gte-multilingual-base models demonstrate strong performance with 0.98 precision, while the indo-bge-m3 model shows high recall but lower precision. From 5 embeddings, Llama Embedding performs significantly worse than the other models. The dot product has higher recall than Euclidean and cosine similarity metrics but embedding model selection significantly influences overall performance. The study underscores the importance of embedding choice and retrieval quality in improving chatbot responses. Further refinements are suggested, including better filtering, reranking retrieved chunks, and fine-tuning the retriever, or LLaMA, to align with the target language domain, such as Bahasa Indonesia. The results show that even though RAG improves how well things are generated, the quality of what is retrieved is still a major problem, which means we need to keep making improvements for the best results in educational uses.

e13609 Background: Precision oncology revolutionized cancer treatment by identifying molecular biomarkers to guide personalized care. The ever-growing body of medical literature presents a challenge for oncologists researching targeted therapies. While recent studies investigated large language models (LLMs) to streamline this process, LLM reliance on general rather than medical knowledge limits clinical relevance and trustworthiness. To address these limitations, we developed a retrieval augmented generation (RAG) system that integrates PubMed clinical studies, trial databases and oncological guidelines with LLMs to support targeted treatment recommendations. The Molecular Tumor Board (MTB) at the Center of Personalized Medicine (ZPMTUM) guided and evaluated treatment options proposed by the LLM to assess their applicability for clinical decision support. Methods: We used 10 publicly accessible fictional patient cases with 7 tumor types and 59 distinct molecular alterations. Our LLM system MEREDITH (Medical Evidence Retrieval and Data Integration for Tailored Healthcare) consists of Google's Gemini Pro, enhanced with RAG and Chain-of-Thought (CoT) prompting. To establish a benchmark, clinical experts at ZPMTUM manually annotated the cases. Informed by MTB expert feedback, we iteratively improved our LLM system from a draft system relying on PubMed-indexed data to an enhanced system, which replicated expert annotation processes by incorporating oncology guidelines, drug availability and trial databases (ClinicalTrials.gov, QuickQueck.de). ZPMTUM assessed credibility and clinical relevance of manually annotated and LLM-generated recommendations. Patient-level data on (likely) pathogenic molecular alterations and recommended treatment options were summarized using median and interquartile range (IQR). Semantic similarity between LLM and clinician responses was assessed using cosine similarity of text vector embeddings; paired t-test evaluated significance. Results: The median of (likely) pathogenic molecular alterations per patient was 2.5 (IQR: 2-3). ZPMTUM identified a median of 2 treatment options per patient (IQR: 1-3), while the enhanced LLM identified a median of 4 (IQR: 3-5). MEREDITH proposed multiple relevant treatment suggestions, including therapies based on preclinical studies, and molecular interactions, for further assessment by the MTB. ZPMTUM prioritized the most suitable clinical option. The mean semantic textual similarity of LLM responses increased significantly from 0.69 in the draft system to 0.76 in the enhanced system (p <0.001). Thus, feedback from ZPMTUM enhanced the model's ability to align its responses with clinician thought processes. Conclusions: Leveraging expert thought processes to instruct LLMs holds promise as a novel decision support tool for precision oncology.

The expansion of generative Artificial Intelligence (AI), particularly Large Language Models (LLMs), has significantly transformed various industries, including law, by automating complex tasks and enhancing overall productivity. However, these models are susceptible to generating fabricated or inaccurate information, commonly referred to as hallucinations. This phenomenon poses substantial risks in fields like legal practice, where precision and accuracy are of utmost importance. To mitigate these challenges, a Retrieval-Augmented Generation (RAG) model integrated with LLMs is proposed. This approach grounds AI-generated responses in verified, authoritative sources, such as legal practice guides and rulebooks, by utilizing a customized vectorized database of legal documents. By doing so, the RAG model effectively minimizes the likelihood of errors in legal advice and case references, thereby improving the accuracy, reliability, and ethical application of AI in legal decision-making processes. This is particularly crucial in high-stakes environments where the cost of misinformation can be significant. In this study, a total of 17 recent Indian legal documents sourced from the official Indian government legislation website were employed to create a comprehensive vector database using an advanced embedding model. Six combinations of core and embedding models were rigorously tested and evaluated to determine their effectiveness. Notably, the combination of Llama2 as the core model and All-mpnet-base-v2 as the embedding model demonstrated superior performance, achieving a BERTScore F1 of 56.87%. This finding underscores the potential of RAG models to enhance the reliability of AI-driven legal assistance, ensuring that users receive contextually relevant and trustworthy legal information.