Oblivionis: A Lightweight Learning and Unlearning Framework for Federated Large Language Models

Large Language Models (LLMs) in Intelligent Computer-Assisted Language Learning enable highly personalized learning, yet raise significant challenges related to pedagogical grounding, data privacy, and instructional validity. Although Knowledge Graphs (KGs) and Federated Learning (FL) can mitigate these issues in isolation, evidence on systematic FL–KG–LLM integration for educational language learning remains limited. This scoping review maps the FL–KG–LLM convergence landscape. Following PRISMA-ScR guidelines, we searched six databases and screened 51 papers (2019–2025) using automated extraction. Our findings indicate limited convergence: no papers integrate all three domains, and 58.8% of approaches remain confined to isolated technological silos. Reporting is also uneven across the corpus, with an average “Not Reported” (NR) rate of 84.5%, most notably for privacy mechanisms (92.2%), validation metrics (90.2%), and Common European Framework of Reference for Languages (CEFR) alignment (88.2%). Domain-specific analysis reveals two distinct patterns: inter-domain gaps (disciplinary silos resulting in expected CEFR absence in single-domain papers) and intra-domain gaps (failure to report domain-critical variables, including 100% parameter NR in FL studies, 86.7% validation NR in KG studies, and 100% CEFR NR in convergence papers). Taken together, these gaps suggest that pedagogical grounding is treated as optional rather than structural. We therefore identify two pillars of pedagogical grounding: a Grounding Pillar, which constrains LLM outputs via Knowledge Graph rules, and a Validation Pillar, which concerns how authoritative frameworks (e.g., CEFR) are mapped onto Knowledge Graph schemas and evaluated. The near-universal absence of CEFR alignment and validation reporting suggests that this second pillar is currently missing, which we term the Integrity Gap—a systematic disconnection between technological innovation and pedagogical grounding inin Intelligent Computer-Assisted Language Learning. By reframing the problem as upstream control and validation, this review informs the design of user-facing automated systems where trust, transparency, and human oversight are critical.

Large language models (LLMs) have made significant advances in the field of natural language processing (NLP). However, their centralized training approach faces challenges related to data privacy, communication efficiency, and robustness against adversarial attacks, particularly in wireless environments. With the gradual depletion of high-quality public data, there is an urgent need to leverage private data distributed across various parties. Although federated learning (FL) offers a privacy-preserving collaborative training paradigm, it struggles to meet the high computational demands of edge devices and remains vulnerable to adversarial attacks. This paper introduces <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ROFED</small>-LLM, a novel framework for robust, privacy-preserving training of LLMs on decentralized private data over wireless networks. By integrating split federated learning, which partitions the model across devices to enhance privacy, with adaptive jamming defense mechanisms, <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ROFED</small>-LLM enables collaborative LLM training without raw data sharing while ensuring resilience against wireless adversarial attacks. Our multi-modal defense strategy combines model-level protections, such as differential privacy and dynamic pruning, with communication-level safeguards, including adaptive beamforming which optimizes wireless signal transmission to mitigate interference, and resource allocation optimization. Extensive experiments across diverse NLP tasks demonstrate <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ROFED</small>-LLM's superiority, achieving a 12.87% improvement in privacy preservation and 18.26% enhancement in jamming resilience compared to existing methods such as FedAvg and SCAFFOLD, with only a marginal 3.94% trade-off in model accuracy. Our code repository has been open sourced at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://anonymous.4open.science/r/RoFed-LLM-54E1</uri>.

In today’s rapidly evolving business landscape, Artificial Intelligence (AI), and specifically Large Language Models (LLMs), are redefining how organizations operate, make decisions, and engage with customers. AI-driven technologies have become indispensable, providing businesses with powerful tools to streamline operations, derive actionable insights from vast data, and foster more meaningful customer interactions. For business leaders, scholars, and practitioners alike, understanding the transformative potential of AI isn’t just advantageous—it’s essential to staying competitive in an increasingly data-driven world.This editorial delves into recent scholarly advancements in LLM applications within business contexts, analyzing studies that explore AI’s potential across various domains, from decision support to creative industries. By introducing a structured framework, this editorial highlights key insights and contributions from recent studies, assessing their value to academia and industry. The following comparative analysis sheds light on how these innovations shape our understanding of AI’s role in business while pointing to future research directions.Puyt and Madsen's (2024) study stands out as a foundational exploration of LLM accuracy, assessing ChatGPT-4's ability to recount the history of the SWOT analysis-a vital business strategy tool. Their findings reveal that, while ChatGPT-4 effectively conveys general concepts, it struggles with detailed historical information, often producing inaccuracies or "hallucinations." This gap underscores the need for LLMs to be trained with verified academic data, particularly for strategic business applications that demand precision. This study not only contributes to the literature by proposing methods to evaluate AI accuracy in historical contexts but also highlights the importance of rigorous information vetting in industry settings where reliability is crucial.In contrast, Raikov et al. (2024) explore a hybrid intelligence model that combines LLM capabilities with explainable AI (XAI) principles to enhance human-machine collaboration. Their approach emphasizes cognitive semantics, improving transparency and decision-making efficiency. The hybrid model's real-time adaptability addresses the needs of complex, regulated industries such as finance and healthcare, where trust in AI decisions is paramount. Academically, this study provides a valuable addition to XAI literature by demonstrating how LLMs can bridge the gap between AI autonomy and human oversight, making it a model for future human-AI interactions in complex business environments.Another significant study by Mariotti and colleagues (2024) examines the integration of LLMs with enterprise knowledge graphs to enhance data-driven decision-making. By enabling organizations to leverage knowledge graphs for more accurate and scalable data retrieval, this research provides a robust framework for businesses seeking efficient knowledge management systems. The academic contribution here lies in advancing the dialogue between LLMs and knowledge graphs, emphasizing ethical data handling and quality standards essential for industry applications. For enterprises, the study offers practical solutions to achieve streamlined data management, balancing automation with privacy and security. 2024) take a different approach, investigating LLMs' role in creative industries, specifically within fashion design. They introduce a hybrid intelligence model that supports creative processes, allowing AI to complement rather than replace human ingenuity. While LLMs in this field demonstrate potential in automating repetitive design tasks and enhancing customer personalization, the study reveals limitations in AI's ability to handle spatial and stylistic nuances. This study's academic contribution lies in promoting human-AI co-creation, inspiring further research into AI applications across diverse creative sectors, including media and marketing.Collectively, these studies not only illuminate LLMs' transformative potential in business but also highlight critical ethical and operational considerations. Ensuring accuracy, transparency, and data privacy are vital to responsibly integrating AI into business workflows. Future research should focus on enhancing LLM accuracy, refining hybrid intelligence models, and exploring creative AI applications, all while maintaining ethical standards. As LLMs evolve, interdisciplinary collaborations will be essential to harness their full potential, making AI an ethical, effective, and innovative force in the business world.

This study examines the benefits of applying federated learning (FL) technology to OPC (Operational Performance Control) systems within industrial automation and data analysis processes. FL enables each production facility to process its data locally while only transmitting model parameters to a central server, thereby preserving data privacy. This approach provides significant advantages in industrial environments, particularly concerning data privacy and communication costs. The study evaluates FL's potential to ensure data privacy, reduce communication costs, improve efficiency in training time, and deliver high performance in predictive maintenance and quality estimation. Model performance was analyzed using accuracy, F1 score, precision, and loss metrics; the results demonstrated that FL achieved a 90% accuracy rate, offering competitive performance compared to centralized modeling. In predictive maintenance and quality analysis specifically, FL achieved 85-88% accuracy while reducing network data load by 65%. These findings validate that FL provides a secure, cost-effective, and efficient solution for industrial data analysis processes by eliminating the need for centralized data collection. In conclusion, FL and OPC integration supports data privacy, cost savings, and communication efficiency in industrial processes. The study highlights that FL could become a prevalent technology in industrial data analysis, establishing a new standard particularly in digital manufacturing processes.

Large Language Models (LLMs) have demonstrated significant potential in Medical Report Generation (MRG), yet their development requires large amounts of medical image-report pairs, which are commonly scattered across multiple centers. Centralizing these data is exceptionally challenging due to privacy regulations, thereby impeding model development and broader adoption of LLM-driven MRG models. To address this challenge, we present FedMRG, the first framework that leverages Federated Learning (FL) to enable privacy-preserving, multi-center development of LLM-driven MRG models, specifically designed to overcome the critical challenge of communication-efficient LLM training under multi-modal data heterogeneity. To start with, our framework tackles the fundamental challenge of communication overhead in federated LLM tuning by employing low-rank factorization to efficiently decompose parameter updates, significantly reducing gradient transmission costs and making LLM-driven MRG feasible in bandwidth-constrained FL settings. Furthermore, we observed the dual heterogeneity in MRG under the FL scenario: varying image characteristics across medical centers, as well as diverse reporting styles and terminology preferences. To address the data heterogeneity, we further enhance FedMRG with (1) client-aware contrastive learning in the MRG encoder, coupled with diagnosis-driven prompts, which capture both globally generalizable and locally distinctive features while maintaining diagnostic accuracy; and (2) a dual-adapter mutual boosting mechanism in the MRG decoder that harmonizes generic and specialized adapters to address variations in reporting styles and terminology. Through extensive evaluation of our established FL-MRG benchmark, we demonstrate the generalizability and adaptability of FedMRG, underscoring its potential in harnessing multi-center data and generating clinically accurate reports while maintaining communication efficiency.

Federated learning-based semantic segmentation for pixel-wise defect detection in additive manufacturing

Privacy and data security have become the new hot topic for regulators in recent years. As a result, Federated Learning (FL) (also called collaborative learning) has emerged as a new training paradigm that allows multiple, geographically distributed nodes to learn a Deep Learning (DL) model together without sharing their data. Blockchain is becoming a new trend as data protection and privacy are concerns in many sectors. Technology is leading the world and transforming into a global village where everything is accessible and transparent. We have presented a blockchain enabled security model using FL that can generate an enhanced DL model without sharing data and improve privacy through higher security and access rights to data. However, existing FL approaches also have unique security vulnerabilities that malicious actors can exploit and compromise the trained model. The FL method is compared to the other known approaches. Users are more likely to choose the latter option, i.e., providing local but private data to the server and using ML apps, performing ML operations on the devices without benefiting from other users’ data, and preventing direct access to raw data and local training of ML models. FL protects data privacy and reduces data transfer overhead by storing raw data on devices and combining locally computed model updates. We have investigated the feasibility of data and model poisoning attacks under a blockchain-enabled FL system built alongside the Ethereum network and the traditional FL system (without blockchain). This work fills a knowledge gap by proposing a transparent incentive mechanism that can encourage good behavior among participating decentralized nodes and avoid common problems and provides knowledge for the FL security literature by investigating current FL systems.

Federated learning (FL) is a distributed model training paradigm that preserves clients’ data privacy. It has gained tremendous attention from both academia and industry. FL hyper-parameters (e.g., the number of selected clients and the number of training passes) significantly affect the training overhead in terms of computation time, transmission time, computation load, and transmission load. However, the current practice of manually selecting FL hyper-parameters imposes a heavy burden on FL practitioners because applications have different training preferences. In this paper, we propose, an automatic FL hyper-parameter tuning algorithm tailored to applications’ diverse system requirements in FL training. iteratively adjusts FL hyper-parameters during FL training and can be easily integrated into existing FL systems. Through extensive evaluations of for diverse applications and FL aggregation algorithms, we show that is lightweight and effective, achieving 8.48%-26.75% system overhead reduction compared to using fixed FL hyper-parameters. This paper assists FL practitioners in designing high-performance FL training solutions. The source code of is available at.

BackgroundThe launch of ChatGPT (OpenAI) in November 2022 attracted public attention and academic interest to large language models (LLMs), facilitating the emergence of many other innovative LLMs. These LLMs have been applied in various fields, including health care. Numerous studies have since been conducted regarding how to use state-of-the-art LLMs in health-related scenarios.ObjectiveThis review aims to summarize applications of and concerns regarding conversational LLMs in health care and provide an agenda for future research in this field.MethodsWe used PubMed, ACM, and the IEEE digital libraries as primary sources for this review. We followed the guidance of PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) to screen and select peer-reviewed research articles that (1) were related to health care applications and conversational LLMs and (2) were published before September 1, 2023, the date when we started paper collection. We investigated these papers and classified them according to their applications and concerns.ResultsOur search initially identified 820 papers according to targeted keywords, out of which 65 (7.9%) papers met our criteria and were included in the review. The most popular conversational LLM was ChatGPT (60/65, 92% of papers), followed by Bard (Google LLC; 1/65, 2% of papers), LLaMA (Meta; 1/65, 2% of papers), and other LLMs (6/65, 9% papers). These papers were classified into four categories of applications: (1) summarization, (2) medical knowledge inquiry, (3) prediction (eg, diagnosis, treatment recommendation, and drug synergy), and (4) administration (eg, documentation and information collection), and four categories of concerns: (1) reliability (eg, training data quality, accuracy, interpretability, and consistency in responses), (2) bias, (3) privacy, and (4) public acceptability. There were 49 (75%) papers using LLMs for either summarization or medical knowledge inquiry, or both, and there are 58 (89%) papers expressing concerns about either reliability or bias, or both. We found that conversational LLMs exhibited promising results in summarization and providing general medical knowledge to patients with a relatively high accuracy. However, conversational LLMs such as ChatGPT are not always able to provide reliable answers to complex health-related tasks (eg, diagnosis) that require specialized domain expertise. While bias or privacy issues are often noted as concerns, no experiments in our reviewed papers thoughtfully examined how conversational LLMs lead to these issues in health care research.ConclusionsFuture studies should focus on improving the reliability of LLM applications in complex health-related tasks, as well as investigating the mechanisms of how LLM applications bring bias and privacy issues. Considering the vast accessibility of LLMs, legal, social, and technical efforts are all needed to address concerns about LLMs to promote, improve, and regularize the application of LLMs in health care.

PurposeLarge language models (LLMs) like Generative Pre-trained Transformer 4 (GPT-4) can assist in detecting errors in radiology reports, but privacy concerns limit their clinical applicability. This study compares closed-source and privacy-compliant open-source LLMs for detecting common errors in radiology reports.Materials and methodsA total of 120 radiology reports were compiled (30 each from X-ray, ultrasound, CT, and MRI). Subsequently, 397 errors from five categories (typographical, numerical, findings-impression discrepancies, omission/insertion, interpretation) were inserted into 100 of these reports; 20 reports were left unchanged. Two open-source models (Llama 3-70b, Mixtral 8x22b) and two commercial closed-source (GPT-4, GPT-4o) were tasked with error detection using identical prompts. The Kruskall–Wallis test and paired t-test were used for statistical analysis.ResultsOpen-source LLMs required less processing time per radiology report than closed-source LLMs (6 ± 2 s vs. 13 ± 4 s; p < 0.001). Closed-source LLMs achieved higher error detection rates than open-source LLMs (GPT-4o: 88% [348/397; 95% CI: 86, 92], GPT-4: 83% [328/397; 95% CI: 80, 87], Llama 3-70b: 79% [311/397; 95% CI: 76, 83], Mixtral 8x22b: 73% [288/397; 95% CI: 68, 77]; p < 0.001). Numerical errors (88% [67/76; 95% CI: 82, 93]) were detected significantly more often than typographical errors (75% [65/86; 95% CI: 68, 82]; p = 0.02), discrepancies between findings and impression (73% [73/101; 95% CI: 67, 80]; p < 0.01), and interpretation errors (70% [50/71; 95% CI: 62, 78]; p = 0.001).ConclusionOpen-source LLMs demonstrated effective error detection, albeit with comparatively lower accuracy than commercial closed-source models, and have potential for clinical applications when deployed via privacy-compliant local hosting solutions.Key PointsQuestionCan privacy-compliant open-source large language models (LLMs) match the error-detection performance of commercial non-privacy-compliant closed-source models in radiology reports?FindingsClosed-source LLMs achieved slightly higher accuracy in detecting radiology report errors than open-source models, with Llama 3-70b yielding the best results among the open-source models.Clinical relevanceOpen-source LLMs offer a privacy-compliant alternative for automated error detection in radiology reports, improving clinical workflow efficiency while ensuring patient data confidentiality. Further refinement could enhance their accuracy, contributing to better diagnosis and patient care.Graphical

Identifying drilling risks for upcoming wells demands insights from offset wells. While some risks may lead to non-productive time (NPT), others remain latent within daily drilling reports (DDRs), which is crucial for comprehensive analysis. Manual extraction from DDRs is often time-consuming. In this paper, we systematically explore the methods to leverage the Large Language Models (LLMs) with enhanced workflows to automate insights extraction and improve risk identification accuracy. We compared LLMs customization approaches and state-of-the-art NLP models to validate LLMs efficiency in the drilling domain. We show that the adaptation of prompt optimization, Retrieval Augmented Generation (RAG) with the pre-trained LLMs before fine-tuning the model, already shows a significant improvement, achieving an accuracy ranging from 80-85%, with precision and recall surging from less than 50%, as observed from our benchmark oil and gas finetuned GPT-2 model (Marlot et al., 2023), to over 70% with our customized model. We addressed the challenge of imbalance training examples during the finetuning of the state-of-the-art GPT-2 model, which caused the underperformance of the benchmark model, particularly in addressing rare drilling event classes. This paper demonstrates LLMs able to improve the generalization and reliability of the imbalance drilling risks, with proper workflow enhancements. Subsequently, we also finetuned both on-premises LLMs (Llama2, Llama3, Mistral), and cloud LLM GPT-3.5 Turbo, which exhibits promising outcomes, achieving better performance to over 90% accuracy, and over 80% precision and recall, compared to the earlier approach using prompt and RAG. This paper contributes to advancing the application of LLMs in the oil and gas industry, for domain-specific tasks, addressing the common challenge of imbalance operation training data, and demonstrating our enhanced workflow outperforms both the benchmark finetuned GPT-2 model, pretrained GPT-4 model, and the finetuned GPT-3.5 Turbo.

Edge-based technologies have emerged as a key enabler to empower low-latency services and incorporate machine learning techniques for learning/inference. However, transferring user data to the edge server to conduct learning could violate data privacy and overburden the network. In addition, the server could receive multiple redundant tasks for inference which leads to redundant computations. In this article, we study both communication and computation issues in edge networks by emphasizing data privacy in a smart home scenario. We design an architecture that incorporates federated edge learning to promote data privacy and a node weighting and dropping scheme to select the appropriate participating devices with quality data and therefore improve the training and reduce communication cost. We further apply Long Short-Term Memory to predict future tasks and proactively store them locally at the edge device. We adopt the computation reuse concept to satisfy incoming tasks with less-to-no computation and thus eliminating redundant computation and further decreasing the computation cost. Simulation results based on real-world dataset show the effectiveness and efficiency of the proposed architecture. The training phase is reached with few iterations, while computation and communication are reduced by up to 80% and 70%, respectively, compared with existing schemes while data privacy is promoted.

Purpose With accelerating advances in artificial intelligence (AI) and digital transformation, the construction domain is rapidly adopting large language models (LLMs) to improve efficiency, safety and sustainability. However, a comprehensive understanding of their applications, adaptation strategies and evaluation practices in this domain remains limited. This paper provides a systematic landscape of LLM research and applications in construction, clarifying their roles, integration pathways and associated challenges. Design/methodology/approach A systematic review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guideline, combining bibliometric and content analysis of publications from 2020 to 2025. The review examines the evolution, implementation strategies and performance characteristics of LLMs in construction domains. Findings LLMs have been increasingly implemented across diverse construction tasks. Most studies utilize pretrained models, adapted through fine-tuning, retrieval-augmented generation or prompt engineering. While LLMs demonstrate strong reasoning and semantic understanding capabilities, challenges persist in domain adaptation, hallucination control, data privacy and lifecycle integration. Practical implications The results provide insights for researchers and practitioners on how LLMs can be effectively implemented throughout the construction domain. The paper highlights the need for developing domain-specific evaluation standards, establishing governance frameworks and aligning LLM applications with existing digital ecosystems such as building information modeling and digital twins. Originality/value This paper presents a comprehensive systematic review focused on LLM applications in construction. By mapping the technological landscape and summarizing key implementation strategies, it establishes a theoretical and practical foundation for advancing intelligent, data-driven and human-centered construction in the era of generative AI.

In decentralized learning scenarios such as federated learning (FL), client data are often highly heterogeneous, leading to non-IID distributions that degrade global model performance, slow convergence, and limit personalization. Conventional FL aggregation schemes inadequately capture client-specific structure and struggle to generalize across diverse data regimes. In this work, we propose a personalized federated learning framework that integrates Fisher Discriminant Analysis (FDA), meta-learning, and convolutional neural networks (CNNs) to jointly enhance class separability, adaptability, and representation quality. FDA is employed to project client-side features into an optimal class-separable latent space, enabling more discriminative modeling of local task characteristics under non-IID conditions. On top of this representation, a meta-learning strategy is used to learn initialization parameters that support fast adaptation, allowing each client to efficiently fine-tune the model to its own data while preserving global knowledge. A CNN backbone is adopted to extract hierarchical feature representations, providing both fine-grained local patterns and robust global semantic embeddings suitable for heterogeneous federated environments. We conduct extensive experiments on CIFAR-10, CIFAR-100, and EMNIST-Writers under non-IID data partitions generated using a Dirichlet concentration parameter α = 0.5. The proposed method achieves 79.6% test accuracy on CIFAR-10, 45.7% on CIFAR-100, and 85.5% on EMNIST-Writers, consistently outperforming strong baseline FL methods in terms of adaptability, generalization, and personalization quality. The results demonstrate that combining class-separable latent spaces with meta-adaptation improves both global coherence and client-specific performance while preserving data privacy and offering good scalability. Future work will extend the framework to multi-modal and sequential data and investigate its deployment in cross-device and cross-silo FL settings. Keywords: federated learning; personalized models; Fisher Discriminant Analysis; meta-learning; non-IID data; data privacy.

Large Language Models (LLMs) became wide-spread during only the last couple of years and are used in almost every scientific and non-scientific domain. Understanding opportunities, applications and limitations of LLMs is crucial for a risk-free, effective and useful implementation of LLMs into scientific workflows. We demonstrate that their effectiveness is maximized not through autonomous operation but through careful integration with specialized tools and contextual knowledge bases.&amp;#160;Using local deployments of modern LLMs (QWen-2.5-coder, LLaMA, Mistral) comes with a number of benefits in a scientific context. Our approach employs vector embeddings for enhanced context retention and metadata databases for structured data access, enabling guided, context-aware interactions with the LLM. Local deployments allow for improved data handling and privacy, improved cost management and a higher degree of customization. Energy consumption can more easily be observed and managed, which can be a crucial property of such a system, especially compared to the newest generation of LLMs, which have extensive power&amp;#160; (and cost) requirements.Opportunities and limitations are explored through two case studies: (1) an LLM-driven system that queries metadata databases to retrieve data from common open data sources and harmonizes patio-temporal subsets into data-cubes, and (2) a VBA-to-Python code translation project to preserve a legacy selection-system forest management software, which was developed in ACCESS / VBA over more than two decades. The LLM's translation process and reasoning are preserved in a vector database for consistent context maintenance and the original as well as the &amp;#8218;new&amp;#8216; code is searchable using the LLM to aid rebuilding a modernized software.&amp;#160;Results suggest that this tool-augmented approach leads to a more reliable and maintainable solution compared to purely LLM-driven implementations, suggesting a new paradigm for integrating AI in scientific workflows where LLMs rather facilitate than replace domain-specific tools and human expertise.