Federated Learning for Medical Image Classification: A Comprehensive Benchmark.

Federated learning (FL), which enables multiple distributed devices (clients) to collaboratively train a global model without transmitting their private data, has attracted much attention in the Internet of Things (IoT) domain. Compared with centralized learning, FL has obvious privacy advantages because it can protect the clients’ raw data from direct access by adversaries. Furthermore, to prevent the adversaries from inferring private information from the transmitted parameters, several FL algorithms based on differential privacy (DP) have been proposed, where the clients add artificial noise to their local parameters for privacy protection. Nevertheless, the added noise would disrupt the learning process and degrade the performance of the trained model. Considering this, in this article, we develop a performance-enhanced DP-based FL (PEDPFL) algorithm, where a classifier-perturbation regularization method is proposed to improve the robustness of the trained model against DP-injected noise. We derive the theoretical privacy and convergence analysis of the proposed algorithm, and also demonstrate the influence of some hyperparameters on the convergence performance. Simulation results on real-world data sets show that the proposed algorithm has better classification performance than the existing DP-based FL algorithms at the same level of privacy protection, and thus, it is more applicable to IoT applications.

In this paper, the convergence time of federated learning (FL), when deployed over a realistic wireless network, is studied. In particular, a wireless network is considered in which wireless users transmit their local FL models (trained using their locally collected data) to a base station (BS). The BS, acting as a central controller, generates a global FL model using the received local FL models and broadcasts it back to all users. Due to the limited number of resource blocks (RBs) in a wireless network, only a subset of users can be selected to transmit their local FL model parameters to the BS at each learning step. Moreover, since each user has unique training data samples, the BS prefers to include all local user FL models to generate a converged global FL model. Hence, the FL training loss and convergence time will be significantly affected by the user selection scheme. Therefore, it is necessary to design an appropriate user selection scheme that can select the users who can contribute toward improving the FL convergence speed more frequently. This joint learning, wireless resource allocation, and user selection problem is formulated as an optimization problem whose goal is to minimize the FL convergence time and the FL training loss. To solve this problem, a probabilistic user selection scheme is proposed such that the BS is connected to the users whose local FL models have significant effects on the global FL model with high probabilities. Given the user selection policy, the uplink RB allocation can be determined. To further reduce the FL convergence time, artificial neural networks (ANNs) are used to estimate the local FL models of the users that are not allocated any RBs for local FL model transmission at each given learning step, which enables the BS to improve the global model, the FL convergence speed, and the training loss. Simulation results show that the proposed approach can reduce the FL convergence time by up to 56% and improve the accuracy of identifying handwritten digits by up to 3%, compared to a standard FL algorithm.

In recent years, Federated Learning (FL) has gained traction as a privacy-centric approach in medical imaging. This study explores the challenges posed by data heterogeneity on FL algorithms, using the COVIDx CXR-3 dataset as a case study. We contrast the performance of the Federated Averaging (FedAvg) algorithm on non-identically and independently distributed (non-IID) data against identically and independently distributed (IID) data. Our findings reveal a notable performance decline with increased data heterogeneity, emphasizing the need for innovative strategies to enhance FL in diverse environments. This research contributes to the practical implementation of FL, extending beyond theoretical concepts and addressing the nuances in medical imaging applications. This research uncovers the inherent challenges in FL due to data diversity. It sets the stage for future advancements in FL strategies to effectively manage data heterogeneity, especially in sensitive fields like healthcare.

Federated Learning (FL) has emerged as a promising approach to collaborative machine learning without the need to share raw data. It enables decentralized model updates while preserving the privacy of each device and reducing the communication overhead. This experiment evaluates the effectiveness of the personalized FL algorithms, namely FedAvg, APPLE, FedBABU and FedProto, in a decentralized setting, with a particular focus on the Fashion MNIST dataset, which is characterized by a non-ideal data distribution. The objective is to identify which algorithm performs optimally in image classification tasks. The experimental results show that both FedProto and APPLE have nearly equivalent and better performance compared to FedBABU and FedAvg. Interestingly, increasing the number of uploads in FedBABU leads to similar results to APPLE and FedProto. However, under limited upload conditions, FedBABU performs similarly to FedAvg. These results provide valuable insights into the differential performance of personalized FL algorithms in non-id data scenarios and provide guidance for their application in distributed environments, especially in sensitive domains such as medical, military and confidential image analysis tasks where privacy and communication efficiency are paramount concerns.

In this paper, the optimization of network performance to support the deployment of federated learning (FL) is investigated. In particular, in the considered model, each user owns a machine learning (ML) model by training through its own dataset, and then transmits its ML parameters to a base station (BS) which aggregates the ML parameters to obtain a global ML model and transmits it to each user. Due to limited radio frequency (RF) resources, the number of users that participate in FL is restricted. Meanwhile, each user uploading and downloading the FL parameters may increase communication costs thus reducing the number of participating users. To this end, we propose to introduce visible light communication (VLC) as a supplement to RF and use compression methods to reduce the resources needed to transmit FL parameters over wireless links so as to further improve the communication efficiency and simultaneously optimize wireless network through user selection and resource allocation. This user selection and bandwidth allocation problem is formulated as an optimization problem whose goal is to minimize the training loss of FL. We first use a model compression method to reduce the size of FL model parameters that are transmitted over wireless links. Then, the optimization problem is separated into two subproblems. The first subproblem is a user selection problem with a given bandwidth allocation, which is solved by a traversal algorithm. The second subproblem is a bandwidth allocation problem with a given user selection, which is solved by a numerical method. The ultimate user selection and bandwidth allocation are obtained by iteratively compressing the model and solving these two subproblems. Simulation results show that the proposed FL algorithm can improve the accuracy of object recognition by up to 16.7% and improve the number of selected users by up to 68.7%, compared to a conventional FL algorithm using only RF.

With the growth of participating clients, the centralized parameter server (PS) will seriously limit the scale and efficiency of Federated Learning (FL). A straightforward approach to scale up the FL system is to construct a Parallel FL (PFL) system with multiple parallel PSes. However, it is unclear whether PFL can really accelerate FL or reduce the training time of FL. Even if the answer is yes, it is non-trivial to design a highly efficient parameter average algorithm for a PFL system. In this paper, we propose a completely parallelizable FL algorithm called P-FedAvg under the PFL architecture. P-FedAvg extends the well-known FedAvg algorithm by allowing multiple PSes to cooperate and train a learning model together. In P-FedAvg, each PS is only responsible for a fraction of total clients, but PSes can mix model parameters in a dedicatedly designed way so that the FL model can well converge. Different from heuristic-based algorithms, P-FedAvg is with theoretical guarantees. To be rigorous, we theoretically analyze the convergence rate of P-FedAvg in terms of the number of conducted iterations, the communication cost of each global iteration and the optimal weights for each PS to mix parameters with its neighbors. Based on theoretical analysis, we conduct a case study on five typical overlay topolgoies formed by PSes to further examine the communication efficiency under different topologies, and investigate how the overlay topology affects the convergence rate, communication cost and robustness of a PFL system. Lastly, we perform extensive experiments with real datasets to verify our analysis and demonstrate that P-FedAvg can significantly speed up FL than traditional FedAvg and other competitive baselines. We believe that our work can help to lay a theoretical foundation for building more efficient PFL systems.

Edge servers and mobile devices are often assigned a large number of computing tasks. However, the data involved in computing tasks is often sensitive in terms of privacy. Our initial proposal is a federated edge learning strategy based on real-world scenarios, which combines blurred data or pseudo shared data. Federated learning is used to train device models with the aim of protecting privacy while enabling mobile devices to more effectively utilize data for decision-making. In the case of limited energy on mobile devices, we propose a federated edge learning algorithm with blurred data sharing. This algorithm can generate more accurate models by uploading partially blurred data. In order to further improve model accuracy and protect privacy of mobile devices, we propose a federated edge learning algorithm with pseudo data sharing based on dataset distillation and generative adversarial networks (GANs) in scenarios with relatively sufficient energy. The experimental results on several traditional datasets show that our proposed algorithms outperform traditional algorithms in terms of accuracy and energy consumption.

Traditional federated learning algorithms suffer from considerable performance reduction with non-identically and independently distributed datasets. This paper proposes a federated learning algorithm based on parallel-ensemble learning, which improves performance for image classification on these datasets. The training process of this algorithm includes basic federation learning and meta federation learning. First, several basic models are trained through cross-validation of federated learning, and then the meta-model is trained using the prediction results of the validation sets. In the training process, the training of different basic models is parallel. In prediction, meta-model is used to aggregate the output of the basic models to get the final prediction results. Our algorithm can achieve higher accuracy than traditional federated learning when using non-independent identically distributed datasets for image classification. Our algorithm aggregates different models through federated learning based on parallel-ensemble method, and improves the image classification performance of federated learning on non-independent identically distributed datasets.

The increasing number and scale of federated learning (FL) jobs necessitates resource efficient scheduling and management of aggregation to make the economics of cloud-hosted aggregation work. Existing FL research has focused on the design of FL algorithms and optimization, and less on aggregation efficacy. In this paper, we propose a new FL aggregation paradigm - "just-in-time" (JIT) aggregation that leverages unique properties of FL jobs, especially the periodicity of model updates, to defer aggregation as much as possible and free compute resources for other FL jobs or other datacenter workloads. We describe a novel way to prioritize FL jobs for aggregation, and demonstrate using multiple datasets, models and FL aggregation algorithms that our techniques can reduce resource usage by 60+% when compared to eager aggregation used in existing FL platforms. We demonstrate that using JIT aggregation has negligible overhead and impact on the latency of the FL job.

Federated learning (FL) has gained attention as a distributed learning paradigm for its data privacy benefits and accelerated convergence through parallel computation. Traditional FL relies on a server-client (SC) architecture, where a central server coordinates multiple clients to train a global model, but this approach faces scalability challenges due to server communication bottlenecks. To overcome this, the ring-all-reduce (RAR) architecture has been introduced, eliminating the central server and achieving bandwidth optimality. However, the tightly coupled nature of RAR's ring topology exposes it to unique Byzantine attack risks not present in SC-based FL. Despite its potential, designing Byzantine-robust RAR-based FL algorithms remains an open problem. To address this gap, we propose BRACE (<u>B</u>yzantine-<u>r</u>obust ring-<u>a</u>ll-redu<u>ce</u>), the first RAR-based FL algorithm to achieve both Byzantine robustness and communication efficiency. We provide theoretical guarantees for the convergence of BRACE under Byzantine attacks, demonstrate its bandwidth efficiency, and validate its practical effectiveness through experiments. Our work offers a foundational understanding of Byzantine-robust RAR-based FL design.

With the continuous development of vehicular ad hoc networks (VANET) security, using federated learning (FL) to deploy intrusion detection models in VANET has attracted considerable attention. Compared to conventional centralized learning, FL retains local training private data, thus protecting privacy. However, sensitive information about the training data can still be inferred from the shared model parameters in FL. Differential privacy (DP) is sophisticated technique to mitigate such attacks. A key challenge of implementing DP in FL is that non-selectively adding DP noise can adversely affect model accuracy, while having many perturbed parameters also increases privacy budget consumption and communication costs for detection models. To address this challenge, we propose FFIDS, a FL algorithm integrating model parameter pruning with differential privacy. It employs a parameter pruning technique based on the Fisher Information Matrix to reduce the privacy budget consumption per iteration while ensuring no accuracy loss. Specifically, FFIDS evaluates parameter importance and prunes unimportant parameters to generate compact sub-models, while recording the positions of parameters in each sub-model. This not only reduces model size to lower communication costs, but also maintains accuracy stability. DP noise is then added to the sub-models. By not perturbing unimportant parameters, more budget can be reserved to retain important parameters for more iterations. Finally, the server can promptly recover the sub-models using the parameter position information and complete aggregation. Extensive experiments on two public datasets and two F2MD simulation datasets have validated the utility and superior performance of the FFIDS algorithm.

Federated and fusion learning methods are state-of-the-art distributed learning approaches which enable model training without collecting private data from users. While federated learning involves lower computation cost as compared to fusion learning, the overall communication cost is higher due to a large number of communication rounds between the clients and the server. On the other hand, fusion learning reduces the overall communication cost by sending distributions of features and model parameters using only one communication round but suffers from high computation cost as it needs to find the distributions of features at the client. This paper presents hybrid fusion learning, a system that leverages hierarchical client-edge-cloud architecture and builds a deep learning model by integrating both fusion and federated learning methods. Our proposed approach uses fusion learning between the client and the edge layer to minimise the communication cost whereas it uses federated learning between the edge and the cloud layer to minimise the computation cost. Our results show that the proposed hybrid fusion learning can significantly reduce the total time taken to train the model with a small drop of around 2% in accuracies as compared to the other two algorithms. Specifically, our results show that fusion and federated learning algorithms take up to 26.28% and 9.74% higher average total time to build the model, respectively, than the proposed hybrid fusion learning approach.

Federated Learning (FL) has recently attracted great interest in sensor-based human activity recognition (HAR) tasks. However, in real-world environment, sensor data on devices is non-independently and identically distributed (Non-IID), e.g., activity data recorded by most devices is sparse, and sensor data distribution for each client may be inconsistent. As a result, the traditional FL methods in the heterogeneous environment may incur a drifted global model that causes slow convergence and a heavy communication burden. Although some FL methods are gradually being applied to HAR, they are designed for overly ideal scenarios and do not address such Non-IID problem in the real-world setting. It is still a question whether they can be applied to cross-device FL. To tackle this challenge, we propose ProtoHAR, a prototype-guided FL framework for HAR, which aims to decouple the representation and classifier in the heterogeneous FL setting efficiently. It leverages the global prototype to correct the activity feature representation to make the prototype knowledge flow among clients without leaking privacy while solving a better classifier to avoid excessive drift of the local model in personalized training. Extensive experiments are conducted on four publicly available datasets: USC-HAD, UNIMIB-SHAR, PAMAP2, and HARBOX, which are collected in both controlled environments and real-world scenarios. The results show that compared with the state-of-the-art FL algorithms, ProtoHAR achieves the best performance and faster convergence speed in HAR datasets.

In modern devices, such as smartphones and IoT, AI hardware implements different ML models to train massive amounts of data for various applications. However, based on the sensitivity of this data, privacy and security concerns, or both, may restrict users from accessing the data storage to conduct the ML training using conventional methods. Federated learning (FL) is consequently emerged to maintain training data distribution among smart mobile devices while aggregating locally processed updates. In addition to improving the model training performance as the number of clients rises, FL also creates a privacy-preserved shared data model. FL execution, however, can be time-consuming. This study proposes a parallelized approach to enhance the performance and privacy of the FL algorithm (FedAvg). In this regard, the FedAvg algorithm is expanded to our proposed model, distributed FedAvg (D-FedAvg), which enables several clients to collaborate concurrently and train a single learning model. To evaluate the performance of the proposed model, we investigated the impact of various numbers of clients and training rounds on the result. To validate our findings, we conducted extensive experiments using MNIST datasets trained using two federated learning models, MLP and CNN. The results show our D-FedAvg can maintain data privacy and dramatically enhance the execution time of FL compared to traditional FedAvg. According to the study, the FL framework's time complexity increases as the number of clients and rounds increases, but parallelization allows it to operate 2–3 times faster than usual.

Secure and efficient federated learning via novel multi-party computation and compressed sensing