Adaptive Personalization in Federated Learning via Feature Subtraction and Generative Decoupling

Artificial intelligence has significantly enhanced disease diagnosis in healthcare, particularly through Deep Learning (DL) and Federated Learning (FL) approaches. These technologies have shown promise in detecting ocular diseases using medical imaging while addressing challenges related to data privacy and security. FL enables collaborative learning without sharing sensitive medical data, making it an attractive solution for healthcare applications. This systematic review aims to analyze the advancements in AI-driven ocular disease detection, with a particular focus on FL-based approaches. The article evaluates the evolution, methodologies, challenges, and effectiveness of FL in enhancing diagnostic accuracy while ensuring data confidentiality. The systematic review followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) framework to ensure transparency and reliability. Research articles published between 2017 and 2024 were identified using academic databases, including Web of Science, Scopus, IEEE Xplore, and PubMed. Studies focusing on DL and FL models for detecting ocular diseases were selected based on predefined inclusion and exclusion criteria. A comparative analysis of the methodologies, architectures, datasets, and performance metrics of different FL models has been presented. The findings indicated that FL preserves data privacy while achieving diagnostic performance comparable to traditional centralized AI models. Various FL models, including FedAvg and FedProx, have been implemented for ocular disease detection, with high accuracy and efficiency. However, challenges, such as data heterogeneity, communication efficiency, and model convergence, persist. FL represents a promising approach for ocular disease detection, balancing diagnostic accuracy with data privacy. Future research may focus on optimizing FL frameworks for improving scalability, communication efficiency, and integrating advanced privacy-preserving techniques.

This study examines the benefits of applying federated learning (FL) technology to OPC (Operational Performance Control) systems within industrial automation and data analysis processes. FL enables each production facility to process its data locally while only transmitting model parameters to a central server, thereby preserving data privacy. This approach provides significant advantages in industrial environments, particularly concerning data privacy and communication costs. The study evaluates FL's potential to ensure data privacy, reduce communication costs, improve efficiency in training time, and deliver high performance in predictive maintenance and quality estimation. Model performance was analyzed using accuracy, F1 score, precision, and loss metrics; the results demonstrated that FL achieved a 90% accuracy rate, offering competitive performance compared to centralized modeling. In predictive maintenance and quality analysis specifically, FL achieved 85-88% accuracy while reducing network data load by 65%. These findings validate that FL provides a secure, cost-effective, and efficient solution for industrial data analysis processes by eliminating the need for centralized data collection. In conclusion, FL and OPC integration supports data privacy, cost savings, and communication efficiency in industrial processes. The study highlights that FL could become a prevalent technology in industrial data analysis, establishing a new standard particularly in digital manufacturing processes.

Federated learning (FL) at the edge is emerging as a cornerstone of next-generation AI, particularly in the era of 5G and 6G connectivity. This chapter explores how FL enables distributed devices and edge servers to collaboratively train AI models across massive IoT (mIoT) environments without centralizing sensitive data. By leveraging edge computing, billions of sensors, smartphones, and smart machines can contribute to powerful global models while preserving privacy and reducing latency. The result is AI that is both scalable and secure, transforming the deluge of decentralized data into actionable intelligence. Real-world applications and trends are highlighted to demonstrate FL’s transformative impact. From smart city sensors and wearable healthcare devices that jointly detect anomalies to 6G base stations coordinating for optimal network performance, FL enables smarter and more resilient systems. The chapter also discusses how this approach addresses challenges such as data privacy, communication costs, and device heterogeneity, making it a critical enabler for digital transformation. As industries embrace connected devices and automation, federated edge learning is poised to drive innovation, unlocking intelligent services, enhancing security, and shaping the future of AI-driven enterprises. Building on this motivation, the chapter progresses from FL foundations to real-world deployment in edge settings, covering key protocols and architectures, including synchronous/asynchronous and hierarchical decentralized approaches. It then surveys system frameworks that jointly optimize learning with energy, communication efficiency, and security, as well as FL-based real-time anomaly detection and intrusion detection systems (IDS) for sensitive Medical IoT applications. Subsequently, it targets 5G/6G intelligence, including spectrum and beam management, Over-The-Air aggregation, and infrastructure learning such as federated radio resource management and D2D-enabled decentralized exchange. Finally, it connects to standards and practice (e.g., ITU-T/3GPP), provides comparison tables, and closes with a gap analysis on scaling, validation, governance, incentives, and interoperability.

Within the federated learning (FL) framework, the client collaboratively trains the model in coordination with a central server, while the training data can be kept locally on the client. Thus, the FL framework mitigates the privacy disclosure and costs related to conventional centralized machine learning. Nevertheless, current surveys indicate that FL still has problems in terms of communication efficiency and privacy risks. In this paper, to solve these problems, we develop an FL framework with communication-efficient and privacy-preserving (FLCP). To realize the FLCP, we design a novel compression algorithm with efficient communication, namely, adaptive weight compression FedAvg (AWC-FedAvg). On the basis of the non-independent and identically distributed (non-IID) and unbalanced data distribution in FL, a specific compression rate is provided for each client, and homomorphic encryption (HE) and differential privacy (DP) are integrated to provide demonstrable privacy protection and maintain the desirability of the model. Therefore, our proposed FLCP smoothly balances communication efficiency and privacy risks, and we prove its security against “honest-but-curious” servers and extreme collusion under the defined threat model. We evaluate the scheme by comparing it with state-of-the-art results on the MNIST and CIFAR-10 datasets. The results show that the FLCP performs better in terms of training efficiency and model accuracy than the baseline method.

Privacy-preserving Federated Learning (FL) based on Differential Privacy (DP) protects clients’ data by adding DP noise to samples’ gradients and has emerged as a de facto standard for data privacy in FL. However, the accuracy of global models in DP-based FL may be reduced significantly when rogue clients occur who deviate from the preset DP-based FL approaches and selfishly inject excessive DP noise beyond expectations, thereby applying a smaller privacy budget in the DP mechanism to ensure a higher level of security. Existing DP-based FL fails to prevent such attacks as they are imperceptible. Under the DP-based FL system and random Gaussian noise, the local model parameters of the rogue clients and the honest clients have identical distributions. In particular, the rogue local models show a low performance, but directly filtering out lower-performance local models compromises the generalizability of global models, as local models trained on scarce data also behave with low performance in the early epoch. In this paper, we propose ReFL, a novel privacy-preserving FL system that enforces DP and avoids the accuracy reduction of global models caused by excessive DP noise of rogue clients. Based on the observation that rogue local models with excessive DP noise and honest local models trained on scarce data have different performance patterns in long-term training epochs, we propose a long-term contribution incentives scheme to evaluate clients’ reputations and identify rogue clients. Furthermore, we design a reputation-based aggregation to avoid the damage of rogue clients’ models on the global model accuracy, based on the incentive reputation. Extensive experiments demonstrate ReFL guarantees the global model accuracy performance 0.77% - 81.71% higher than existing DP-based FL methods in the presence of rogue clients.

With the growth of participating clients, the centralized parameter server (PS) will seriously limit the scale and efficiency of Federated Learning (FL). A straightforward approach to scale up the FL system is to construct a Parallel FL (PFL) system with multiple parallel PSes. However, it is unclear whether PFL can really accelerate FL or reduce the training time of FL. Even if the answer is yes, it is non-trivial to design a highly efficient parameter average algorithm for a PFL system. In this paper, we propose a completely parallelizable FL algorithm called P-FedAvg under the PFL architecture. P-FedAvg extends the well-known FedAvg algorithm by allowing multiple PSes to cooperate and train a learning model together. In P-FedAvg, each PS is only responsible for a fraction of total clients, but PSes can mix model parameters in a dedicatedly designed way so that the FL model can well converge. Different from heuristic-based algorithms, P-FedAvg is with theoretical guarantees. To be rigorous, we theoretically analyze the convergence rate of P-FedAvg in terms of the number of conducted iterations, the communication cost of each global iteration and the optimal weights for each PS to mix parameters with its neighbors. Based on theoretical analysis, we conduct a case study on five typical overlay topolgoies formed by PSes to further examine the communication efficiency under different topologies, and investigate how the overlay topology affects the convergence rate, communication cost and robustness of a PFL system. Lastly, we perform extensive experiments with real datasets to verify our analysis and demonstrate that P-FedAvg can significantly speed up FL than traditional FedAvg and other competitive baselines. We believe that our work can help to lay a theoretical foundation for building more efficient PFL systems.

The open radio access network (O-RAN) is designed to support the diverse wireless services for beyond 5th-generation (B5G) mobile networks. However, this also expands the potential attack surface, necessitating improved mechanisms for detecting cyberattacks. Advanced artificial intelligence (AI) algorithms, in conjunction with RAN intelligent controllers (RICs), can be utilized to identify threats such as distributed denial-of-service (DDoS) attacks. Nevertheless, AI introduces significant data privacy concern. To address these issues, secure federated learning (FL) can be leveraged to locally train cyberattack detection models and securely transmit the model data for aggregation, thus ensuring protection against eavesdropping. Moreover, peer-to-peer (P2P) FL can be used to avoid the single point of failure inherent in centralized FL. However, securing P2P FL with encryption/decryption or secure average computation (SAC) can result in high communication costs that do not scale well with the number of FL clients. In this paper, we propose a novel P2P FL strategy that ensures secure operation while significantly reducing communication costs. Specifically, we integrate client selection and transfer learning within the RIC-based P2P FL system to detect cyberattacks. Our experiments demonstrate the performance of our method across various scenarios with both balanced and unbalanced dataset distributions. We highlight its superiority in terms of accuracy, robustness, and cost compared to existing benchmarks. Furthermore, we extend our evaluation to a 5G O-RAN testbed, assessing the system’s efficiency, accuracy, and adaptability under real-time independent and non-independent and identically distributed (IID/non-IID) traffic conditions. This includes analyzing communication cost, execution time, model loss, and live traffic testing results for practical and real-time deployment.

The federated learning (FL) paradigm is well-suited for the field of medical image analysis, as it can effectively cope with machine learning on isolated multi-center data while protecting the privacy of participating parties. However, current research on optimization algorithms in FL often focuses on limited datasets and scenarios, primarily centered around natural images, with insufficient comparative experiments in medical contexts. In this work, we conduct a comprehensive evaluation of several state-of-the-art FL algorithms in the context of medical imaging. We conduct a fair comparison of classification models trained using various FL algorithms across multiple medical imaging datasets. Additionally, we evaluate system performance metrics, such as communication cost and computational efficiency, while considering different FL architectures. Our findings show that medical imaging datasets pose substantial challenges for current FL optimization algorithms. No single algorithm consistently delivers optimal performance across all medical FL scenarios, and many optimization algorithms may under-perform when applied to these datasets. Our experiments provide a benchmark and guidance for future research and application of FL in medical imaging contexts. Furthermore, we propose an efficient and robust method that combines generative techniques using denoising diffusion probabilistic models with label smoothing to augment datasets, widely enhancing the performance of FL on classification tasks across various medical imaging datasets.

Federated learning is a popular distributed machine learning paradigm that enables collaborative model training at multiple entities via exchanging intermediate learning results. Security and communication efficiency are crucial for successful applications of federated learning in various privacy-sensitive services. However, existing work focused on gradient defense and communication efficiency separately, and also incurred additional computation, signaling, and accuracy overhead. A lightweight (in terms of time-complexity and signaling) technique that simultaneously achieves security and communication efficiency is critical for massive resource-constrained devices (e.g., Internet-of-Things generating the data), but has yet to be established. This paper proposes a secure and efficient federated learning framework with provable communication-accuracy-security performance guarantees. A low-complexity and signaling-free stochastic quantization module is added at the client side that quantizes the original local gradients to discrete values for communication-efficient global aggregation. The stochastic quantization module is shown to be interpreted as triangular or Gaussian-multiply-triangular noises under uniform or Gaussian distributions of local gradients, hence protecting data privacy. We prove that the proposed framework exhibits an { <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (log <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> 1/δ), <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (δ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ), <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (1/δ)}-tradeoff between the communication overhead, model accuracy, and data protection, where δ is an adjustable quantization interval. Experimental results validate the tradeoff and the superiority of the proposed stochastic quantization technique in terms of communication efficiency (only 14.1% of differential privacy and 0.2% of homomorphic encryption) and computation complexity (similar to differential privacy and only 0.03% of homomorphic encryption). Under the same data protection performance, the proposed approach also outperforms (in terms of accuracy) differential privacy in all the 9 comparison settings on CIFAR10 dataset.

Federated Learning (FL) is considered the key, enabling approach for privacy-preserving, distributed machine learning (ML) systems. FL requires the periodic transmission of ML models from users to the server. Therefore, communication via resource-constrained networks is currently a fundamental bottleneck in FL, which is restricting the ML model complexity and user participation. One of the notable trends to reduce the communication cost of FL systems is gradient compression, in which techniques in the form of sparsification are utilized. However, these methods utilize a single compression rate for all users and do not consider communication heterogeneity in a real-world FL system. Therefore, these methods are bottlenecked by the worst communication capacity across users. Further, sparsification methods are non-adaptive and do not utilize the redundant, similar information across users' ML models for compression. In this paper, we introduce a novel Dynamic Sparsification for Federated Learning (DSFL) approach that enables users to compress their local models based on their communication capacity at each iteration by using two novel sparsification methods: layer-wise similarity sparsification (LSS) and extended top- <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$K$</tex> sparsification. LSS enables DSFL to utilize the global redundant information in users' models by using the Centralized Kernel Alignment (CKA) similarity for sparsification. The extended top- <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$K$</tex> model sparsification method empowers DSFL to accommodate the heterogeneous communication capacity of user devices by allowing different values of sparsification rate <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$K$</tex> for each user at each iteration. Our extensive experimental results <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> All code and experiments are publicly available at: https://github.com/mahdibeit/DSFL. on three datasets show that DSFL has a faster convergence rate than fixed sparsification, and as the communication heterogeneity increases, this gap increases. Further, our thorough experimental investigations uncover the similarities of user models across the FL system.

Normalization is an important but understudied challenge in privacy-related application domains such as federated learning (FL), differential privacy (DP), and differentially private federated learning (DP-FL). While the unsuitability of batch normalization for these domains has already been shown, the impact of other normalization methods on the performance of federated or differentially private models is not well-known. To address this, we draw a performance comparison among layer normalization (LayerNorm), group normalization (GroupNorm), and the recently proposed kernel normalization (KernelNorm) in FL, DP, and DP-FL settings. Our results indicate LayerNorm and GroupNorm provide no performance gain compared to the baseline (i.e. no normalization) for shallow models in FL and DP. They, on the other hand, considerably enhance the performance of shallow models in DP-FL and deeper models in FL and DP. KernelNorm, moreover, significantly outperforms its competitors in terms of accuracy and convergence rate (or communication efficiency) for both shallow and deeper models in all considered learning environments. Given these key observations, we propose a kernel normalized ResNet architecture called KNResNet-13 for differentially private learning. Using the proposed architecture, we provide new state-of-the-art accuracy values on the CIFAR-10 and Imagenette datasets, when trained from scratch.

With its privacy preservation and communication efficiency, federated learning (FL) has emerged as a promising learning framework for beyond 5G wireless networks. It is anticipated that future wireless networks will jointly serve both FL and downlink non-FL user groups in the same time-frequency resource. While in the downlink of each FL iteration, both groups simultaneously receive data from the base station in the same time-frequency resource, the uplink of each FL iteration requires bidirectional communication to support uplink transmission for FL users and downlink transmission for non-FL users. To overcome this challenge, we present half-duplex (HD) and full-duplex (FD) communication schemes to serve both groups. More specifically, we adopt the massive multiple-input multiple-output technology and aim to maximize the minimum effective rate of non-FL users under a quality of service (QoS) latency constraint for FL users. Since the formulated problem is nonconvex, we propose a power control algorithm based on successive convex approximation to find a stationary solution. Numerical results show that the proposed solutions perform significantly better than the considered baselines schemes. Moreover, the FD-based scheme outperforms the HD-based counterpart in scenarios where the self-interference is small or moderate and/or the size of FL model updates is large.

Additive manufacturing (AM) enables the production of customized and complex metal parts, providing greater flexibility than traditional mass production. However, maintaining consistent product quality for AM processes remains challenging due to process variability and heat emission fluctuations, which can cause defects such as overheating. Overheating occurs when non-uniform temperature distributes across the printed parts, leading to issues such as structural weaknesses. Predicting heat emissions accurately is essential to preventing such issues, but AM facilities, particularly small- and medium-sized enterprises (SMEs), generate limited and heterogeneous datasets due to unique configurations and privacy concerns, making traditional centralized predictive model training paradigms impractical. Federated learning (FL) offers a privacy-preserving solution, it alleviates data privacy and data scarcity concerns by enabling collaborative learning across decentralized datasets while keeping data local. However, its performance suffers under heterogeneous client data and varied computational resources. This creates a need to address data heterogeneity while maintaining both global model performance and individual client personalization. We propose FedScope-KD, a novel heterogeneity-aware FL framework designed to balance the scope of global and personalized learning. Client heterogeneity level is quantified prior to the FL training, to improve the model weight aggregation during training. This is achieved through component decomposition and knowledge distillation. The framework decomposes client data into process-invariant representations, which capture the fundamental nature of the laser powder bed fusion (LPBF) process, and process-variant representations, which reflect local, client-specific configuration settings. This decomposition ensures that global models benefit from collaborative learning on the homogeneous data representations, while personalized predictions remain tailored to client-specific heterogeneous data representations. A knowledge distillation (KD) mechanism further enhances the framework by efficiently transferring process-invariant knowledge from a global teacher model to local student models. We compare the heterogeneity index generation and calculate the Dynamic Time Warping (DTW) distance for both inter-client variant and invariant components. The variant components exhibit greater distances between clients, reflecting their inherent heterogeneity, while the invariant components are more closely aligned across clients. This pattern demonstrates the effectiveness of the decomposition process in separating heterogeneous and homogeneous information. The heterogeneity index is further used for weight aggregation during FL training. The framework’s performance is validated through a comparison with baseline FL methods such as FedAvg, as well as centralized and individual learning paradigms. Performance metrics include personalized performance and global accuracy with MSE. Through experiments on heat emission datasets in the LPBF process, we show that FedScope-KD successfully addresses the challenges of data heterogeneity and resource constraints, providing a robust solution of predictive modeling for SMEs.

This article provides a comprehensive survey of aggregation strategies in federated learning (FL). This decentralized machine learning (ML) paradigm enables multiple clients to collaboratively train models without sharing their local datasets. Aggregation is a pivotal aspect of FL, as it integrates model updates from diverse clients into a unified global model while addressing critical challenges such as data heterogeneity, scalability, and privacy preservation. The study categorizes aggregation strategies into three primary approaches: data-centric, model-centric, and secure aggregation, each tailored to address distinct problems in FL systems. Data-centric strategies focus on addressing non-independent and identically distributed (non-IID) data across clients, ensuring that model updates account for imbalances in data distributions. Model-centric strategies optimize the aggregation of model parameters, emphasizing techniques such as weighted averaging and model distillation to improve model performance across clients with diverse data characteristics. Secure aggregation techniques aim to enhance privacy and robustness, protecting client data from potential adversarial threats through encryption-based methods like secure multi-party computation (SMPC) and Byzantine-resilient techniques. The analysis delves into the advantages and limitations of these aggregation strategies, particularly their role in tackling challenges like non-IID data, communication efficiency, and resistance to adversarial attacks. Furthermore, the article identifies existing research gaps in FL, including the need for more scalable and robust aggregation methods capable of reducing communication costs, enhancing privacy guarantees, and improving performance in highly heterogeneous environments. These insights provide a roadmap for future research aimed at advancing aggregation strategies in FL to improve model accuracy, security, and efficiency in real-world applications.

The fusion of Federated Learning (FL) and deep generative models is transforming medical imaging by enabling privacy-preserving and data-efficient machine learning. Training large-scale deep models on radiological imaging data remains challenging due to data scarcity, heterogeneity, and strict privacy constraints that limit data sharing across institutions. FL addresses these challenges by employing collaborative model training that does not expose raw patient data, while generative models synthesize realistic medical images to alleviate scarcity and imbalance. The convergence of these two fields has given rise to federated generative models (FGMs), which extend Generative Adversarial Networks (GANs), Variational Autoencoders (VAE), and Diffusion models into distributed and privacy-preserving environments. FGMs have demonstrated significant potential in various medical imaging tasks, including data augmentation, image reconstruction, cross-modality conversion, and enhancing classification and segmentation models. Despite promising progress, the field remains in its early stages, facing open challenges in communication efficiency, scalability, data heterogeneity, and clinical reliability. This survey provides a comprehensive review of FGMs in medical imaging, analyzing their architectures, applications, and benchmark datasets, while highlighting key challenges and outlining future research directions to advance privacy-preserving generative modeling in healthcare.