Contrastive encoder pre-training-based clustered federated learning for heterogeneous data

Privacy and data security have become the new hot topic for regulators in recent years. As a result, Federated Learning (FL) (also called collaborative learning) has emerged as a new training paradigm that allows multiple, geographically distributed nodes to learn a Deep Learning (DL) model together without sharing their data. Blockchain is becoming a new trend as data protection and privacy are concerns in many sectors. Technology is leading the world and transforming into a global village where everything is accessible and transparent. We have presented a blockchain enabled security model using FL that can generate an enhanced DL model without sharing data and improve privacy through higher security and access rights to data. However, existing FL approaches also have unique security vulnerabilities that malicious actors can exploit and compromise the trained model. The FL method is compared to the other known approaches. Users are more likely to choose the latter option, i.e., providing local but private data to the server and using ML apps, performing ML operations on the devices without benefiting from other users’ data, and preventing direct access to raw data and local training of ML models. FL protects data privacy and reduces data transfer overhead by storing raw data on devices and combining locally computed model updates. We have investigated the feasibility of data and model poisoning attacks under a blockchain-enabled FL system built alongside the Ethereum network and the traditional FL system (without blockchain). This work fills a knowledge gap by proposing a transparent incentive mechanism that can encourage good behavior among participating decentralized nodes and avoid common problems and provides knowledge for the FL security literature by investigating current FL systems.

Federated learning (FL)-a distributed machine learning that offers collaborative training of global models across multiple clients. FL has been considered for the design and development of many FL systems in various domains. Hence, we present a comprehensive survey and analysis of existing FL systems, drawing insights from more than 250 articles published in 2019-2024. Our review elucidates the functioning of FL systems, particularly in comparison with alternative distributed learning approaches. Considering the healthcare domain as an example, we define the building blocks of a typical FL healthcare system, including system architecture, federation scale, data partitioning, open-source frameworks, ML models, and aggregation algorithms. Furthermore, we identify and discuss key challenges associated with the design and implementation of FL systems within the healthcare sector while outlining the directions of future research. In general, through systematic categorization and analysis of existing FL systems, we offer insights to design efficient, accurate, and privacy-preserving healthcare applications using cutting-edge FL techniques.

Federated learning (FL) is a distributed model training paradigm that preserves clients’ data privacy. It has gained tremendous attention from both academia and industry. FL hyper-parameters (e.g., the number of selected clients and the number of training passes) significantly affect the training overhead in terms of computation time, transmission time, computation load, and transmission load. However, the current practice of manually selecting FL hyper-parameters imposes a heavy burden on FL practitioners because applications have different training preferences. In this paper, we propose, an automatic FL hyper-parameter tuning algorithm tailored to applications’ diverse system requirements in FL training. iteratively adjusts FL hyper-parameters during FL training and can be easily integrated into existing FL systems. Through extensive evaluations of for diverse applications and FL aggregation algorithms, we show that is lightweight and effective, achieving 8.48%-26.75% system overhead reduction compared to using fixed FL hyper-parameters. This paper assists FL practitioners in designing high-performance FL training solutions. The source code of is available at.

Federated Learning (FL) is considered the key, enabling approach for privacy-preserving, distributed machine learning (ML) systems. FL requires the periodic transmission of ML models from users to the server. Therefore, communication via resource-constrained networks is currently a fundamental bottleneck in FL, which is restricting the ML model complexity and user participation. One of the notable trends to reduce the communication cost of FL systems is gradient compression, in which techniques in the form of sparsification are utilized. However, these methods utilize a single compression rate for all users and do not consider communication heterogeneity in a real-world FL system. Therefore, these methods are bottlenecked by the worst communication capacity across users. Further, sparsification methods are non-adaptive and do not utilize the redundant, similar information across users' ML models for compression. In this paper, we introduce a novel Dynamic Sparsification for Federated Learning (DSFL) approach that enables users to compress their local models based on their communication capacity at each iteration by using two novel sparsification methods: layer-wise similarity sparsification (LSS) and extended top- <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$K$</tex> sparsification. LSS enables DSFL to utilize the global redundant information in users' models by using the Centralized Kernel Alignment (CKA) similarity for sparsification. The extended top- <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$K$</tex> model sparsification method empowers DSFL to accommodate the heterogeneous communication capacity of user devices by allowing different values of sparsification rate <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$K$</tex> for each user at each iteration. Our extensive experimental results <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> All code and experiments are publicly available at: https://github.com/mahdibeit/DSFL. on three datasets show that DSFL has a faster convergence rate than fixed sparsification, and as the communication heterogeneity increases, this gap increases. Further, our thorough experimental investigations uncover the similarities of user models across the FL system.

Utilizing the federated learning (FL) technique, data owners can collaboratively train artificial intelligence models, retaining all training data on their premises to minimize the potential for personal data breaches. However, self-interested users (e.g., free riders) bring new challenges that hinder the development of FL techniques. To this end, we propose a two-stage client selection scheme comprising a multiarmed bandit (MAB)-based candidate client selection method and an auction-based training client selection method. Specifically, our client selection scheme initially formulates the FL system into an MAB system, where clients are the arms and the server is the player. Then, we quantify the similarity between a local model and the server side, which is the designed metric for model aggregation and reward computation updating based on the fuzzy mathematical strategy. Next, based on the Thompson Sampling strategy, the server can intelligently determine the reward of each client, and clients with more significant rewards have the chance for local model training. With an auction method, the server can determine the training clients to reduce the training cost while maximizing each client’s revenue. Extensive experiments on real-world data sets demonstrate that the proposed scheme outperforms representative FL schemes (i.e., FedAvg, FedProx, FedMax, and MFL) regarding the model’s convergence rate and cost in FL systems with free riders.

Federated learning (FL) is a promising technology for achieving privacy-preserving edge intelligence and has attracted extensive attention from industry and academia. However, in the FL training process, the server directly aggregates local models from mobile devices, which poses serious privacy and security threats. The identity authentication mechanism can provide FL with local model integrity and source authentication. However, the existing schemes are centralized, and most of them are computationally expensive, resulting in limited performance. To address these issues, this paper proposes a decentralized and lightweight anonymous FL identity authentication scheme, namely DAFL. In our scheme, we first design a decentralized and simplified storage FL authentication framework by combining the directed acyclic graph (DAG) blockchain and accumulator. Then, we propose a lightweight digital signature algorithm that supports batch verification for authentication. Finally, nodes interact through pseudonyms to achieve anonymous communication, and the trusted authority (TA) can track and recover the real identities of nodes when malicious behavior occurs. We theoretically prove the security of the proposed DAFL. The extensive experiments demonstrate that DAFL achieves lower authentication overhead and better convergence performance compared to existing authentication schemes and vanilla FL systems.

• Trustworthy AI is a prerequisite for deploying FL in high-risk, multi-stakeholder domains. • Privacy-preserving guarantees alone are insufficient to sustain trust in agentic and dynamic FL systems. • We present a requirement-driven taxonomy of challenges aligned with Trustworthy AI principles, explicitly extended to agentic AI and LLM-enabled settings. • We present a coordination blueprint maps Trustworthy AI requirements to FL-native controls and decision-centric, auditable evidence across the lifecycle. • We reframe trust in FL as a continuously maintained operating condition and introduce coordination tools to operationalize it. • We propose Trust Report 2.0 to operationalize trust as evidence, not assumptions, by surfacing what decisions were made, why, and under which constraints, without centralizing raw data. Federated Learning (FL) enables privacy-preserving collaborative learning, yet deployments increasingly show that privacy guarantees alone do not sustain trust in high-risk settings. As FL systems move toward agentic AI, large language model–enabled, and dynamically adaptive architectures, trustworthiness becomes a system-level problem shaped by autonomous decision-making, non-stationary environments, and multi-stakeholder governance. We argue for Trustworthy FL (TFL), treating trust as a continuously maintained operating condition rather than a static model property. Through the lens of Trust Report 2.0, we propose a requirement-driven taxonomy of challenges grounded in TAI and explicitly extended to account for control-plane decisions, agency, and system dynamics across the federated lifecycle. Building on this diagnosis, we introduce a coordination blueprint that structures cross-requirement trade-offs, decision justification, and governance alignment in TFL systems. To operationalize assurance, Trust Report 2.0 is instantiated as a lightweight, privacy-preserving artifact that surfaces decision-centric trust evidence without centralizing raw data. We illustrate applicability via healthcare as a stress-test domain, focusing on oncology FL under regulatory pressure and clinical risk.

Federated learning (FL) is an emerging machine learning paradigm. With FL, distributed data owners aggregate their model updates to train a shared deep neural network collaboratively, while keeping the training data locally. However, FL has little control over the local data and the training process. Therefore, it is susceptible to poisoning attacks, in which malicious or compromised clients use malicious training data or local updates as the attack vector to poison the trained global model. Moreover, the performance of existing detection and defense mechanisms drops significantly in a scaled-up FL system with non-iid data distributions. In this paper, we propose a defense scheme named CONTRA to defend against poisoning attacks, e.g., label-flipping and backdoor attacks, in FL systems. CONTRA implements a cosine-similarity-based measure to determine the credibility of local model parameters in each round and a reputation scheme to dynamically promote or penalize individual clients based on their per-round and historical contributions to the global model. With extensive experiments, we show that CONTRA significantly reduces the attack success rate while achieving high accuracy with the global model. Compared with a state-of-the-art (SOTA) defense, CONTRA reduces the attack success rate by 70% and reduces the global model performance degradation by 50%.

Federated Learning (FL) is a collaborative machine learning (ML) framework that combines on-device training and server-based aggregation to train a common ML model among distributed agents. In this work, we propose an asynchronous FL design with periodic aggregation to tackle the straggler issue in FL systems. Considering limited wireless communication resources, we investigate the effect of different scheduling policies and aggregation designs on the convergence performance. Driven by the importance of reducing the bias and variance of the aggregated model updates, we propose a scheduling policy that jointly considers the channel quality and training data representation of user devices. The effectiveness of our channel-aware data-importance-based scheduling policy, compared with state-of-the-art methods proposed for synchronous FL, is validated through simulations. Moreover, we show that an ``age-aware'' aggregation weighting design can significantly improve the learning performance in an asynchronous FL setting.

Recent medical applications are largely dominated by the application of Machine Learning (ML) models to assist expert decisions, leading to disruptive innovations in radiology, pathology, genomics, and hence modern healthcare systems in general. Despite the profitable usage of AI-based algorithms, these data-driven methods are facing issues such as the scarcity and privacy of user data, as well as the difficulty of institutions exchanging medical information. With insufficient data, ML is prevented from reaching its full potential, which is only possible if the database consists of the full spectrum of possible anatomies, pathologies, and input data types. To solve these issues, Federated Learning (FL) appeared as a valuable approach in the medical field, allowing patient data to stay where it is generated. Since an FL setting allows many clients to collaboratively train a model while keeping training data decentralized, it can protect privacy-sensitive medical data. However, FL is still unable to deliver all its promises and meets the more stringent requirements (e.g., latency, security) of a healthcare system based on multiple Internet of Medical Things (IoMT). For example, although no data are shared among the participants by definition in FL systems, some security risks are still present and can be considered as vulnerabilities from multiple aspects. This paper sheds light upon the emerging deployment of FL, provides a broad overview of current approaches and existing challenges, and outlines several directions of future work that are relevant to solving existing problems in federated healthcare, with a particular focus on security and privacy issues.

This article provides a comprehensive survey of aggregation strategies in federated learning (FL). This decentralized machine learning (ML) paradigm enables multiple clients to collaboratively train models without sharing their local datasets. Aggregation is a pivotal aspect of FL, as it integrates model updates from diverse clients into a unified global model while addressing critical challenges such as data heterogeneity, scalability, and privacy preservation. The study categorizes aggregation strategies into three primary approaches: data-centric, model-centric, and secure aggregation, each tailored to address distinct problems in FL systems. Data-centric strategies focus on addressing non-independent and identically distributed (non-IID) data across clients, ensuring that model updates account for imbalances in data distributions. Model-centric strategies optimize the aggregation of model parameters, emphasizing techniques such as weighted averaging and model distillation to improve model performance across clients with diverse data characteristics. Secure aggregation techniques aim to enhance privacy and robustness, protecting client data from potential adversarial threats through encryption-based methods like secure multi-party computation (SMPC) and Byzantine-resilient techniques. The analysis delves into the advantages and limitations of these aggregation strategies, particularly their role in tackling challenges like non-IID data, communication efficiency, and resistance to adversarial attacks. Furthermore, the article identifies existing research gaps in FL, including the need for more scalable and robust aggregation methods capable of reducing communication costs, enhancing privacy guarantees, and improving performance in highly heterogeneous environments. These insights provide a roadmap for future research aimed at advancing aggregation strategies in FL to improve model accuracy, security, and efficiency in real-world applications.

Federated Learning (FL) is a Machine Learning (ML) paradigm in which multiple devices collaboratively train a model without sharing their local data. This decentralized approach provides significant privacy benefits, enabling compliance with data protection regulations and safeguarding sensitive user information by keeping raw data on local devices. Instead of transmitting raw data, FL sends model updates to a central aggregator to improve the global model. However, this process can result in higher Carbon Dioxide (CO₂) emissions compared to traditional centralized ML systems, due to the increased number of participating devices and communication rounds. This study evaluates the performance, convergence speed, energy efficiency, and environmental impact of FL models compared to centralized models, using the Modified National Institute of Standards and Technology dataset (MNIST) and Canadian Institute for Advanced Research-10 classes dataset (CIFAR-10). Four models were tested: two FL models and two centralized models. The evaluation focused on accuracy, number of training rounds to convergence, and total CO₂ emissions. To optimize both convergence and energy efficiency, a dynamic hill-climbing-based early stopping technique was introduced. After every 100 rounds, model accuracy improvements were assessed, and training was terminated early if further gains fell below a shrinking threshold, effectively reducing unnecessary computation and energy consumption. Results show that, under the tested conditions, FL models achieved competitive or higher accuracy than centralized models, particularly on non-Independent and Identically Distributed (IID) data distributions. For example, the federated MNIST model reached 98.79% accuracy with a significantly lower carbon footprint when early stopping was applied. Overall, the proposed optimization approach reduced CO₂ emissions by approximately 60% without substantial loss in accuracy. By integrating privacy preservation, explicit regulatory relevance, and a practical dynamic optimization method, this research demonstrates that FL can deliver strong model performance while meeting modern requirements for data privacy and environmental sustainability

Federated learning (FL) is an attractive industrial paradigm to accomplish distributed artificial intelligence (AI) training collaboratively in a data privacy-preserving manner. Most existing designs for FL systems assume that industrial user equipments (UEs) participate voluntarily in FL training. However, since both AI model training and transmission consume considerable energy, UEs are reluctant to participate without economic rewards. Hence, the lack of proper economic reward incentive mechanism results in low UE utility and frustrates UEs' enthusiasm for participating in training. To address the above challenge, in this article, we propose a two-phase energy-aware reward incentive mechanism for the edge-cloud-assisted hierarchical federated learning (HFL) system to optimize the overall UE utility, thereby, incentivizing UEs to participate more actively. Specifically, at the cloud server phase, we design an energy quantity-aware incentive mechanism for reasonably distributing rewards to its sub-edge-assisted FL systems. Subsequently, at the edge server phase, based on the quantitative analysis for the optimal reward allocation solution, we develop an energy-aware water filling-based reward incentive mechanism to adapt to individual needs of UEs and maximize the overall UE utility. Experiments verify that, compared to well-known benchmarks, our incentive mechanism can improve the overall UE utility by up to 55.94% and better incentivize UEs to participate in training.

Accelerating Federated Learning with genetic algorithm enhancements

Federated learning (FL) protects data privacy by enabling distributed model training without direct access to client data. However, its distributed nature makes it vulnerable to model and data poisoning attacks. While numerous defenses filter malicious clients using statistical metrics, they overlook the role of model redundancy, where not all parameters contribute equally to the model and attack performance. Current attacks manipulate all model parameters uniformly, making them more detectable, while defenses focus on the overall statistics of client updates, leaving gaps for more sophisticated attacks. We propose an attack-agnostic augmentation method to enhance the stealthiness and effectiveness of existing poisoning attacks in FL, exposing flaws in current defenses and highlighting the need for fine-grained FL security. Our three-stage methodology, including pill construction, pill poisoning, and pill injection, injects poison into a compact subnet (i.e., pill) of the global model during the iterative FL training. Experimental results show that FL poisoning attacks enhanced by our method can bypass 8 state-of-the-art (SOTA) defenses, gaining an up to 7x error rate increase, as well as on average a more than 2x error rate increase on both IID and non-IID data, in both cross-silo and cross-device FL systems.