DSFL: Dynamic Sparsification for Federated Learning

Privacy and data security have become the new hot topic for regulators in recent years. As a result, Federated Learning (FL) (also called collaborative learning) has emerged as a new training paradigm that allows multiple, geographically distributed nodes to learn a Deep Learning (DL) model together without sharing their data. Blockchain is becoming a new trend as data protection and privacy are concerns in many sectors. Technology is leading the world and transforming into a global village where everything is accessible and transparent. We have presented a blockchain enabled security model using FL that can generate an enhanced DL model without sharing data and improve privacy through higher security and access rights to data. However, existing FL approaches also have unique security vulnerabilities that malicious actors can exploit and compromise the trained model. The FL method is compared to the other known approaches. Users are more likely to choose the latter option, i.e., providing local but private data to the server and using ML apps, performing ML operations on the devices without benefiting from other users’ data, and preventing direct access to raw data and local training of ML models. FL protects data privacy and reduces data transfer overhead by storing raw data on devices and combining locally computed model updates. We have investigated the feasibility of data and model poisoning attacks under a blockchain-enabled FL system built alongside the Ethereum network and the traditional FL system (without blockchain). This work fills a knowledge gap by proposing a transparent incentive mechanism that can encourage good behavior among participating decentralized nodes and avoid common problems and provides knowledge for the FL security literature by investigating current FL systems.

Federated learning is a collaborative/distributed machine learning system which is designed to address the privacy issues in centralized machine learning systems. The transparency and provenance of a machine learning model are important aspects of federated learning systems since they impact peoples’ lives in various domains (e.g., from healthcare to personal finance to employment). However, most of the existing federated learning systems deal with centralized coordinators which are vulnerable to attacks and privacy breaches. Also, they do not provide any standard transparency and provenance mechanisms for the resulting models. In this paper, we propose a blockchain and Model Card-based integrated federated learning system "Bassa-ML" providing enhanced transparency and trust for the models. Model parameter sharing, local model generation, model averaging, and model sharing functions are implemented using smart contracts. The generated models, model training information, and model reports are stored in the blockchain ledger as Model Card Objects. This results in enhanced transparency and auditability to the federated learning process.

Federated learning (FL)-a distributed machine learning that offers collaborative training of global models across multiple clients. FL has been considered for the design and development of many FL systems in various domains. Hence, we present a comprehensive survey and analysis of existing FL systems, drawing insights from more than 250 articles published in 2019-2024. Our review elucidates the functioning of FL systems, particularly in comparison with alternative distributed learning approaches. Considering the healthcare domain as an example, we define the building blocks of a typical FL healthcare system, including system architecture, federation scale, data partitioning, open-source frameworks, ML models, and aggregation algorithms. Furthermore, we identify and discuss key challenges associated with the design and implementation of FL systems within the healthcare sector while outlining the directions of future research. In general, through systematic categorization and analysis of existing FL systems, we offer insights to design efficient, accurate, and privacy-preserving healthcare applications using cutting-edge FL techniques.

Federated learning (FL) is an emerging machine learning paradigm. With FL, distributed data owners aggregate their model updates to train a shared deep neural network collaboratively, while keeping the training data locally. However, FL has little control over the local data and the training process. Therefore, it is susceptible to poisoning attacks, in which malicious or compromised clients use malicious training data or local updates as the attack vector to poison the trained global model. Moreover, the performance of existing detection and defense mechanisms drops significantly in a scaled-up FL system with non-iid data distributions. In this paper, we propose a defense scheme named CONTRA to defend against poisoning attacks, e.g., label-flipping and backdoor attacks, in FL systems. CONTRA implements a cosine-similarity-based measure to determine the credibility of local model parameters in each round and a reputation scheme to dynamically promote or penalize individual clients based on their per-round and historical contributions to the global model. With extensive experiments, we show that CONTRA significantly reduces the attack success rate while achieving high accuracy with the global model. Compared with a state-of-the-art (SOTA) defense, CONTRA reduces the attack success rate by 70% and reduces the global model performance degradation by 50%.

Utilizing the federated learning (FL) technique, data owners can collaboratively train artificial intelligence models, retaining all training data on their premises to minimize the potential for personal data breaches. However, self-interested users (e.g., free riders) bring new challenges that hinder the development of FL techniques. To this end, we propose a two-stage client selection scheme comprising a multiarmed bandit (MAB)-based candidate client selection method and an auction-based training client selection method. Specifically, our client selection scheme initially formulates the FL system into an MAB system, where clients are the arms and the server is the player. Then, we quantify the similarity between a local model and the server side, which is the designed metric for model aggregation and reward computation updating based on the fuzzy mathematical strategy. Next, based on the Thompson Sampling strategy, the server can intelligently determine the reward of each client, and clients with more significant rewards have the chance for local model training. With an auction method, the server can determine the training clients to reduce the training cost while maximizing each client’s revenue. Extensive experiments on real-world data sets demonstrate that the proposed scheme outperforms representative FL schemes (i.e., FedAvg, FedProx, FedMax, and MFL) regarding the model’s convergence rate and cost in FL systems with free riders.

Federated learning is a collaborative and distributed machine learning model that addresses the privacy issues in centralized machine learning models. It emerges as a promising technique that addresses the data sharing concerns for data-private multi-institutional collaborations. However, most existing federated learning systems deal with centralized coordinators and are vulnerable to attacks and privacy breaches. We propose a blockchain-empowered coordinator-less decentralized, federated learning platform “Rahasak-ML” to solve issues in centralized coordinator-based federated learning systems by providing better transparency and trust. It uses an incremental learning approach to train the model by multiple peers in the blockchain network. Rahasak-ML is integrated into the Rahasak blockchain as its data analytics and machine learning platform. Each peer in the blockchain can establish supervised or unsupervised machine learning models with the existing data on its own off-chain storage. Once a peer generates a model, it can be incrementally/continuously trained and aggregated by other peers through the blockchain using the federated learning approach without requiring a centralized coordinator. The model parameters sharing, local model generation, incremental model training, and model sharing functions are implemented in the Rahasak-ML platform. We discussed the promise of Rahasak-ML machine learning in medicine.KeywordsFederated learningBlockchainMedicineDrug discoveryBig data

Blockchain network (BCN)-based Self-Sovereign Identity (SSI) has emerged lately as an identity and access management framework that is based on Distributed Ledger Technology (DLT) and allows users to control their own data. Federated Learning (FL), on the other hand, provides a collaborative framework to update Machine Learning (ML) models without relying explicitly on data exchange between the users. This paper investigates identity management and authentication for vehicle users in the context of FL. We propose a novel approach based on blockchain-based SSI, which focuses on maintaining the authenticity and integrity of vehicle users’ identities and data exchanged between the users and the aggregation server during the execution of the FL iterations. A primary objective of this paper is to achieve shorter durations for credential operations in an FL setting as the system size scales out. Integrating BCN-based SSI into the FL framework addresses several critical FL challenges, ensuring enhanced system security and operational integrity. This synergy of BCN-based SSI with federated learning enables robust identity verification providing a solution to fundamental trustworthiness issues in FL without sacrificing the benefits of decentralized data control, improving both the performance and reliability of the FL system. Experimental results suggest that the proposed FL-based system, together with credential management on a blockchain platform, has the potential to significantly improve data integrity and ensure the authentication of users. More specifically, the results of the FL system demonstrate that it takes longer (on the order of a hundred seconds) as the number of rounds and clients increase, while the implemented Decentralized Identifier (DID) system relying on BCN-based SSI has dramatically shorter dedicated time for completing credential operations.

Federated learning (FL) is a machine learning framework where multiple distributed edge internet-of-things (IoT) devices collaboratively train a model under the orchestration of a central server while keeping the training data distributed on the IoT devices. FL can mitigate the privacy risks and costs from data collection in traditional centralized machine learning. However, the deployment of standard FL is hindered by the expense of the communication of the gradients from the devices to the server. Hence, many gradient compression methods have been proposed to reduce the communication cost. However, the existing methods ignore the structural correlations of the gradients, and therefore lead to a large compression loss which will decelerate the training convergence. Moreover, many of the existing compression schemes do not enable over-the-air aggregation, and hence require huge communication resources. In this work, we propose a gradient compression scheme, named FedOComp, which leverages the correlations of the stochastic gradients in FL systems for efficient compression of the high-dimension gradients with over-the-air aggregation. The proposed design can achieve a smaller deceleration of the training convergence compared to other gradient compression methods since the compression kernel exploits the structural correlations of the gradients. It also directly enables over-the-air aggregation to save communication resources. The derived convergence analysis and simulation results further illustrate that under the same power cost, the proposed scheme has a much faster convergence rate and higher test accuracy compared to existing baselines.

Federated Learning (FL) is a collaborative machine learning (ML) framework that combines on-device training and server-based aggregation to train a common ML model among distributed agents. In this work, we propose an asynchronous FL design with periodic aggregation to tackle the straggler issue in FL systems. Considering limited wireless communication resources, we investigate the effect of different scheduling policies and aggregation designs on the convergence performance. Driven by the importance of reducing the bias and variance of the aggregated model updates, we propose a scheduling policy that jointly considers the channel quality and training data representation of user devices. The effectiveness of our channel-aware data-importance-based scheduling policy, compared with state-of-the-art methods proposed for synchronous FL, is validated through simulations. Moreover, we show that an ``age-aware'' aggregation weighting design can significantly improve the learning performance in an asynchronous FL setting.

Federated Learning (FL) is a paradigm in distributed machine learning, which has gained significant attention in the recent years especially in the domain of Internet of Things. Federated Learning saves communication bandwidth when compared to centralized machine learning process and is also considered to be privacy preserving as the raw data at the clients need not be transmitted to the server for the FL learning. Recent studies on FL have exposed privacy vulnerabilities particularly in the form of Gradient Leakage Attacks (GLA) in which the adversaries can generate training data of the client from shared gradients or model updates from clients. The available solutions in the literature for GLA based on perturbing the gradients from clients leads to a drop in the performance of the FL system while attempting to preserve privacy. We propose HAFedL, an improved novel hessian aware adaptive privacy preserving FL scheme in which the performance of the model is not significantly affected due to the aspects introduced in the FL architecture to improve the privacy of the model against the GLA. The HAFedL is also robust to the data heterogeneity and device heterogeneity (particularly straggler effect) which may be present in the clients participating in the FL. The performance of HAFedL is tested for two applications- IoT device identification and digit classification. The proposed HAFedL scheme can be utilized in privacy sensitive domains such as smart city applications, Industrial Internet of Things, Internet of Robotic Things, Internet of Medical Things, etc.

MobFedLS: A framework to provide federated learning for mobile nodes in V2X environments

Recent medical applications are largely dominated by the application of Machine Learning (ML) models to assist expert decisions, leading to disruptive innovations in radiology, pathology, genomics, and hence modern healthcare systems in general. Despite the profitable usage of AI-based algorithms, these data-driven methods are facing issues such as the scarcity and privacy of user data, as well as the difficulty of institutions exchanging medical information. With insufficient data, ML is prevented from reaching its full potential, which is only possible if the database consists of the full spectrum of possible anatomies, pathologies, and input data types. To solve these issues, Federated Learning (FL) appeared as a valuable approach in the medical field, allowing patient data to stay where it is generated. Since an FL setting allows many clients to collaboratively train a model while keeping training data decentralized, it can protect privacy-sensitive medical data. However, FL is still unable to deliver all its promises and meets the more stringent requirements (e.g., latency, security) of a healthcare system based on multiple Internet of Medical Things (IoMT). For example, although no data are shared among the participants by definition in FL systems, some security risks are still present and can be considered as vulnerabilities from multiple aspects. This paper sheds light upon the emerging deployment of FL, provides a broad overview of current approaches and existing challenges, and outlines several directions of future work that are relevant to solving existing problems in federated healthcare, with a particular focus on security and privacy issues.

With the growth of participating clients, the centralized parameter server (PS) will seriously limit the scale and efficiency of Federated Learning (FL). A straightforward approach to scale up the FL system is to construct a Parallel FL (PFL) system with multiple parallel PSes. However, it is unclear whether PFL can really accelerate FL or reduce the training time of FL. Even if the answer is yes, it is non-trivial to design a highly efficient parameter average algorithm for a PFL system. In this paper, we propose a completely parallelizable FL algorithm called P-FedAvg under the PFL architecture. P-FedAvg extends the well-known FedAvg algorithm by allowing multiple PSes to cooperate and train a learning model together. In P-FedAvg, each PS is only responsible for a fraction of total clients, but PSes can mix model parameters in a dedicatedly designed way so that the FL model can well converge. Different from heuristic-based algorithms, P-FedAvg is with theoretical guarantees. To be rigorous, we theoretically analyze the convergence rate of P-FedAvg in terms of the number of conducted iterations, the communication cost of each global iteration and the optimal weights for each PS to mix parameters with its neighbors. Based on theoretical analysis, we conduct a case study on five typical overlay topolgoies formed by PSes to further examine the communication efficiency under different topologies, and investigate how the overlay topology affects the convergence rate, communication cost and robustness of a PFL system. Lastly, we perform extensive experiments with real datasets to verify our analysis and demonstrate that P-FedAvg can significantly speed up FL than traditional FedAvg and other competitive baselines. We believe that our work can help to lay a theoretical foundation for building more efficient PFL systems.

Federated learning enables privacy-preserving and data-efficient dimension prediction and part qualification across additive manufacturing factories

Federated learning (FL) has arisen as an alternative to collecting large amounts of data in a central place to train a machine learning (ML) model. FL is privacy-friendly, allowing multiple parties to collaboratively train an ML model without exchanging or transmitting their training data. For this purpose, an aggregator iteratively coordinates the training process among parties, and parties simply share with the aggregator model updates, which contain information pertinent to the model such as neural network weights. Besides privacy, generalization has been another key driver for FL: parties who do not have enough data to train a good performing model by themselves can now engage in FL to obtain an ML model suitable for their tasks. Products and real applications in the industry and consumer space have demonstrated the power of this learning paradigm. Recently, foundation models have taken the AI community by storm, promising to solve the shortage of labeled data. A foundation model is a powerful model that can be recycled for a variety of use cases by applying techniques such as zero-shot learning and full or parameter-efficient fine tuning. The premise is that the amount of data required to fine tune a foundation model for a new task is much smaller than fully training a traditional model from scratch. The reason why this is the case is that a good foundation model has already learned relevant general representations, and thus, adapting it to a new task only requires a minimal number of additional samples. This raises the question: Is FL still alive in the era of foundation models? In this talk, I will address this question. I will present some use cases where FL is very much alive. In these use cases, finding a foundation model with a desired representation is difficult if not impossible. With this pragmatic point of view, I hope to shed some light into a real use case where disparate private data is available in isolation at different parties and where labels may be located at a single party that doesn’t have any other information, making it impossible for a single party to train a model on its own. Furthermore, in some vertically-partitioned scenarios, cleaning data is not an option due to privacy-related reasons and it is not clear how to apply foundation models. Finally, I will also go over a few other requirements that are often overlooked, such as unlearning of data and its implications for the lifecycle management of FL and systems based on foundation models.