Sparse Personalized Federated Learning.

Federated learning (FL) is gradually becoming a key learning paradigm in Privacy-preserving Machine Learning (ML) systems. In FL, a large number of clients cooperate with a central server to learn a shared model without sharing their own data sets. However, since there is a great disparity between the client data sets, standard FL is often hard to tune and suffers from performance degradation due to the inharmony among local models. To this end, in this paper we propose a novel FL scheme, termed client reputation federated learning (CRFL), which dynamically assesses the reputation of the clients participating in FL. Our method leverages techniques from model explanation, and aims at precisely measure each client's impact to the global model. To be specific, we first calculate the saliency-weighted variance on pixelwise relevance scores as the quality factor of a single sample. Then we extract activation function values at the last hidden layer to compute the divergence factor of individual data set. Finally, the server integrates these two factors as an assessment of the client reputation. By leveraging such assessment, CRFL can dynamically adjust the weights of the clients in each aggregation round, thus leading to a significant improvement over the baseline method in terms of model accuracy and convergence rate. Intensive experiments are conducted on the MNIST and CIFAR-10 data sets, and experimental results demonstrate the efficacy of the proposed method.

As a novel distributed machine learning approach, federated learning (FL) is proposed to train a global model while preserving data privacy. However, some studies manifest that adversaries can still recover private information from the gradients. Differential privacy (DP) is a rigorous mathematical tool to protect records in a database against leakage. It has been widely applied in FL by perturbing the gradients. Nevertheless, while using DP in FL, the convergence performance of the global model is inevitably degraded. In this paper, we implement a DP-based FL scheme, which achieves local DP (LDP) by adding well-designed Gaussian noise on the gradients before clients upload them to the server. After that, we propose two strategies to improve the convergence performance of the DP-based FL. Both methods are realized by modifying the local objective function to limit the effect of LDP noise on convergence without degrading the privacy protection level. We then provide the detailed framework which adopts the LDP scheme and two strategies. The framework on different machine learning models is tested by simulation results, which show that our framework can improve the convergence performance up to 40% faster under different noise compared with other DP-based FL. Finally, we show the theoretical convergence guarantee of our proposed framework by first presenting the expected decrease in the global loss function for one round of training and then providing the upper convergence bound after multiple communication rounds.

Mobile and Web-of-Things (WoT) devices at the network edge account for more than half of the world's web traffic, making a great data source for various machine learning (ML) applications, particularly federated learning (FL) which offers a promising solution to privacy-preserving ML feeding on these data. FL allows edge mobile and WoT devices to train a shared global ML model under the orchestration of a central parameter server. In the real world, due to resource heterogeneity, these edge devices often train different versions of models (e.g., VGG-16 and VGG-19) or different ML models (e.g., VGG and ResNet) for the same ML task (e.g., computer vision and speech recognition). Existing FL schemes have assumed that participating edge devices share a common model architecture, and thus cannot facilitate FL across edge devices with heterogeneous ML model architectures. We explored this architecture heterogeneity challenge and found that FL can and should accommodate these edge devices to improve model accuracy and accelerate model training. This paper presents our findings and FlexiFed, a novel scheme for FL across edge devices with heterogeneous model architectures, and three model aggregation strategies for accommodating architecture heterogeneity under FlexiFed. Experiments with four widely-used ML models on four public datasets demonstrate 1) the usefulness of FlexiFed; and 2) that compared with the state-of-the-art FL scheme, FlexiFed improves model accuracy by 2.6%-9.7% and accelerates model convergence by 1.24 × -4.04 ×.

As distributed data sources grow, and privacy protection becomes more important, federated learning (FL) is becoming an important way to model by collaboration under data privacy constraints. However, existing federated learning schemes still suffer from data privacy preservation and security problems. We propose a federated learning scheme that also helps better preserve privacy and defend against Byzantine attacks. This type of scheme is based on discrete stochastic noise, using multi-layer privacy preservation to protect private data and defend against Byzantine attacks. First, we propose a dual-layer protection mechanism of adding discrete noise and differential privacy noise, both of which aim at protecting against inference attacks and data privacy. Second, we introduce a dual server architecture to separate aggregation and privacy processing to mitigate the server collusion risk. Lastly, we use a secure aggregation defense strategy to mitigate the negative effect of Byzantine nodes on the global model. It is also shown that our improved scheme performs better than the original LSFL scheme in terms of its inference resistance and privacy protection.

In this paper, the convergence time of federated learning (FL), when deployed over a realistic wireless network, is studied. In particular, a wireless network is considered in which wireless users transmit their local FL models (trained using their locally collected data) to a base station (BS). The BS, acting as a central controller, generates a global FL model using the received local FL models and broadcasts it back to all users. Due to the limited number of resource blocks (RBs) in a wireless network, only a subset of users can be selected to transmit their local FL model parameters to the BS at each learning step. Moreover, since each user has unique training data samples, the BS prefers to include all local user FL models to generate a converged global FL model. Hence, the FL training loss and convergence time will be significantly affected by the user selection scheme. Therefore, it is necessary to design an appropriate user selection scheme that can select the users who can contribute toward improving the FL convergence speed more frequently. This joint learning, wireless resource allocation, and user selection problem is formulated as an optimization problem whose goal is to minimize the FL convergence time and the FL training loss. To solve this problem, a probabilistic user selection scheme is proposed such that the BS is connected to the users whose local FL models have significant effects on the global FL model with high probabilities. Given the user selection policy, the uplink RB allocation can be determined. To further reduce the FL convergence time, artificial neural networks (ANNs) are used to estimate the local FL models of the users that are not allocated any RBs for local FL model transmission at each given learning step, which enables the BS to improve the global model, the FL convergence speed, and the training loss. Simulation results show that the proposed approach can reduce the FL convergence time by up to 56% and improve the accuracy of identifying handwritten digits by up to 3%, compared to a standard FL algorithm.

Privacy and data security have become the new hot topic for regulators in recent years. As a result, Federated Learning (FL) (also called collaborative learning) has emerged as a new training paradigm that allows multiple, geographically distributed nodes to learn a Deep Learning (DL) model together without sharing their data. Blockchain is becoming a new trend as data protection and privacy are concerns in many sectors. Technology is leading the world and transforming into a global village where everything is accessible and transparent. We have presented a blockchain enabled security model using FL that can generate an enhanced DL model without sharing data and improve privacy through higher security and access rights to data. However, existing FL approaches also have unique security vulnerabilities that malicious actors can exploit and compromise the trained model. The FL method is compared to the other known approaches. Users are more likely to choose the latter option, i.e., providing local but private data to the server and using ML apps, performing ML operations on the devices without benefiting from other users’ data, and preventing direct access to raw data and local training of ML models. FL protects data privacy and reduces data transfer overhead by storing raw data on devices and combining locally computed model updates. We have investigated the feasibility of data and model poisoning attacks under a blockchain-enabled FL system built alongside the Ethereum network and the traditional FL system (without blockchain). This work fills a knowledge gap by proposing a transparent incentive mechanism that can encourage good behavior among participating decentralized nodes and avoid common problems and provides knowledge for the FL security literature by investigating current FL systems.

Additive manufacturing (AM) enables the production of customized and complex metal parts, providing greater flexibility than traditional mass production. However, maintaining consistent product quality for AM processes remains challenging due to process variability and heat emission fluctuations, which can cause defects such as overheating. Overheating occurs when non-uniform temperature distributes across the printed parts, leading to issues such as structural weaknesses. Predicting heat emissions accurately is essential to preventing such issues, but AM facilities, particularly small- and medium-sized enterprises (SMEs), generate limited and heterogeneous datasets due to unique configurations and privacy concerns, making traditional centralized predictive model training paradigms impractical. Federated learning (FL) offers a privacy-preserving solution, it alleviates data privacy and data scarcity concerns by enabling collaborative learning across decentralized datasets while keeping data local. However, its performance suffers under heterogeneous client data and varied computational resources. This creates a need to address data heterogeneity while maintaining both global model performance and individual client personalization. We propose FedScope-KD, a novel heterogeneity-aware FL framework designed to balance the scope of global and personalized learning. Client heterogeneity level is quantified prior to the FL training, to improve the model weight aggregation during training. This is achieved through component decomposition and knowledge distillation. The framework decomposes client data into process-invariant representations, which capture the fundamental nature of the laser powder bed fusion (LPBF) process, and process-variant representations, which reflect local, client-specific configuration settings. This decomposition ensures that global models benefit from collaborative learning on the homogeneous data representations, while personalized predictions remain tailored to client-specific heterogeneous data representations. A knowledge distillation (KD) mechanism further enhances the framework by efficiently transferring process-invariant knowledge from a global teacher model to local student models. We compare the heterogeneity index generation and calculate the Dynamic Time Warping (DTW) distance for both inter-client variant and invariant components. The variant components exhibit greater distances between clients, reflecting their inherent heterogeneity, while the invariant components are more closely aligned across clients. This pattern demonstrates the effectiveness of the decomposition process in separating heterogeneous and homogeneous information. The heterogeneity index is further used for weight aggregation during FL training. The framework’s performance is validated through a comparison with baseline FL methods such as FedAvg, as well as centralized and individual learning paradigms. Performance metrics include personalized performance and global accuracy with MSE. Through experiments on heat emission datasets in the LPBF process, we show that FedScope-KD successfully addresses the challenges of data heterogeneity and resource constraints, providing a robust solution of predictive modeling for SMEs.

Federated learning (FL) is a distributed model training paradigm that preserves clients’ data privacy. It has gained tremendous attention from both academia and industry. FL hyper-parameters (e.g., the number of selected clients and the number of training passes) significantly affect the training overhead in terms of computation time, transmission time, computation load, and transmission load. However, the current practice of manually selecting FL hyper-parameters imposes a heavy burden on FL practitioners because applications have different training preferences. In this paper, we propose, an automatic FL hyper-parameter tuning algorithm tailored to applications’ diverse system requirements in FL training. iteratively adjusts FL hyper-parameters during FL training and can be easily integrated into existing FL systems. Through extensive evaluations of for diverse applications and FL aggregation algorithms, we show that is lightweight and effective, achieving 8.48%-26.75% system overhead reduction compared to using fixed FL hyper-parameters. This paper assists FL practitioners in designing high-performance FL training solutions. The source code of is available at.

We study federated edge learning, where a global model is trained collaboratively using privacy-sensitive data at the edge of a wireless network. A parameter server (PS) keeps track of the global model and shares it with the wireless edge devices for training using their private local data. The devices then transmit their local model updates, which are used to update the global model, to the PS. The algorithm, which involves transmission over PS-to-device and device-to-PS links, continues until the convergence of the global model or lack of any participating devices. In this study, we consider device selection based on downlink channels over which the PS shares the global model with the devices. Performing digital downlink transmission, we design a partial device participation framework where a subset of the devices is selected for training at each iteration. Therefore, the participating devices can have a better estimate of the global model compared to the full device participation case which is due to the shared nature of the broadcast channel with the price of updating the global model with respect to a smaller set of data. At each iteration, the PS broadcasts different quantized global model updates to different participating devices based on the last global model estimates available at the devices. We investigate the best number of participating devices through experimental results for image classification using the MNIST dataset with biased distribution.

Edge computing is widely recognized as a crucial technology for the upcoming generation of communication networks and has garnered significant interest from both industry and academia. Compared to other offloading models like cloud computing, it provides faster data processing capabilities, enhanced security measures, and lower costs by leveraging the proximity of the edge servers to the end devices. This helps mitigate the privacy concerns associated with data transfer in edge computing, by reducing the distance between the data source and the server. Raw data in typical edge computing scenarios still need to be sent to the edge server, leading to data leakage and privacy breaches. Federated Learning (FL) is a distributed model training paradigm that preserves end devices’ data privacy. Therefore, it is crucial to incorporate FL into edge computing to protect data privacy. However, the high training overhead of FL makes it impractical for edge computing. In this study, we propose to facilitate the integration of FL and edge computing by optimizing FL hyper-parameters, which can significantly reduce FL’s training overhead and make it more affordable for edge computing.

Artificial intelligence (AI) has, in the recent past, experienced a rebirth with the growth of generative AI systems such as ChatGPT and Bard. These systems are trained with billions of parameters and have enabled widespread accessibility and understanding of AI among different user groups. Widespread adoption of AI has led to the need for understanding how machine learning (ML) models operate to build trust in them. An understanding of how these models generate their results remains a huge challenge that explainable AI seeks to solve. Federated learning (FL) grew out of the need to have privacy-preserving AI by having ML models that are decentralized but still share model parameters with a global model. This study sought to examine the extent of development of the explainable AI field within the FL environment in relation to the main contributions made, the types of FL, the sectors it is applied to, the models used, the methods applied by each study, and the databases from which sources are obtained. A systematic search in 8 electronic databases, namely, Web of Science Core Collection, Scopus, PubMed, ACM Digital Library, IEEE Xplore, Mendeley, BASE, and Google Scholar, was undertaken. A review of 26 studies revealed that research on explainable FL is steadily growing despite being concentrated in Europe and Asia. The key determinants of FL use were data privacy and limited training data. Horizontal FL remains the preferred approach for federated ML, whereas post hoc explainability techniques were preferred. There is potential for development of novel approaches and improvement of existing approaches in the explainable FL field, especially for critical areas. OSF Registries 10.17605/OSF.IO/Y85WA; https://osf.io/y85wa.

Federated learning (FL) enables the training of machine learning models on decentralized data, potentially improving data privacy. However, the FL distributed architecture is vulnerable to poisoning attacks. In this paper, we propose an FL method capable of mitigating these attacks through a triad of defense strategies: organizing clients into groups, evaluating the local performance of global models during training, and using a voting scheme during the inference phase. The proposed approach first divides the clients into randomly sampled groups, each generating a distinct global model. Each client trains a local model on their private data and submits it to the central server. The central server aggregates the local models within each group to generate the global models. Then, each client receives all global models, selects the best performing one as their new local model, and the process repeats until training is complete. During the inference phase, each client classifies its inputs according to a majority-based voting scheme among the global models. Our experiments using the HAR and MNIST datasets demonstrate that our method can effectively mitigate poisoning attacks without compromising the global model's performance.

Federated Learning (FL) is a machine learning approach that enables multiple users to share their local models for the aggregation of a global model, protecting data privacy by avoiding the sharing of raw data. However, frequent parameter sharing between users and the aggregator can incur high risk of membership privacy leakage. In this paper, we propose LiPFed, a computationally lightweight privacy preserving FL scheme using secure decentralized aggregation for edge networks. Under this scheme, we ensure privacy preservation on the aggregation side, and promote lightweight computation on the user side. By incorporating blockchain and additive secret sharing algorithm, we effectively protect the membership privacy of both local models and global models. Furthermore, the secure decentralized aggregation mechanism safeguards against potential compromises of the aggregator. Meanwhile, smart contract is introduced to identify malicious models uploaded by edge nodes and return trustworthy global models to users. Rigorous security analysis shows the effectiveness of this scheme in privacy preservation. Extensive experiments verify that LiPFed outperforms the state-of-the-art schemes in terms of training efficiency, model accuracy, and privacy preservation.

Artificial intelligence has significantly enhanced disease diagnosis in healthcare, particularly through Deep Learning (DL) and Federated Learning (FL) approaches. These technologies have shown promise in detecting ocular diseases using medical imaging while addressing challenges related to data privacy and security. FL enables collaborative learning without sharing sensitive medical data, making it an attractive solution for healthcare applications. This systematic review aims to analyze the advancements in AI-driven ocular disease detection, with a particular focus on FL-based approaches. The article evaluates the evolution, methodologies, challenges, and effectiveness of FL in enhancing diagnostic accuracy while ensuring data confidentiality. The systematic review followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) framework to ensure transparency and reliability. Research articles published between 2017 and 2024 were identified using academic databases, including Web of Science, Scopus, IEEE Xplore, and PubMed. Studies focusing on DL and FL models for detecting ocular diseases were selected based on predefined inclusion and exclusion criteria. A comparative analysis of the methodologies, architectures, datasets, and performance metrics of different FL models has been presented. The findings indicated that FL preserves data privacy while achieving diagnostic performance comparable to traditional centralized AI models. Various FL models, including FedAvg and FedProx, have been implemented for ocular disease detection, with high accuracy and efficiency. However, challenges, such as data heterogeneity, communication efficiency, and model convergence, persist. FL represents a promising approach for ocular disease detection, balancing diagnostic accuracy with data privacy. Future research may focus on optimizing FL frameworks for improving scalability, communication efficiency, and integrating advanced privacy-preserving techniques.

Federated Learning (FL) is a paradigm in distributed machine learning, which has gained significant attention in the recent years especially in the domain of Internet of Things. Federated Learning saves communication bandwidth when compared to centralized machine learning process and is also considered to be privacy preserving as the raw data at the clients need not be transmitted to the server for the FL learning. Recent studies on FL have exposed privacy vulnerabilities particularly in the form of Gradient Leakage Attacks (GLA) in which the adversaries can generate training data of the client from shared gradients or model updates from clients. The available solutions in the literature for GLA based on perturbing the gradients from clients leads to a drop in the performance of the FL system while attempting to preserve privacy. We propose HAFedL, an improved novel hessian aware adaptive privacy preserving FL scheme in which the performance of the model is not significantly affected due to the aspects introduced in the FL architecture to improve the privacy of the model against the GLA. The HAFedL is also robust to the data heterogeneity and device heterogeneity (particularly straggler effect) which may be present in the clients participating in the FL. The performance of HAFedL is tested for two applications- IoT device identification and digit classification. The proposed HAFedL scheme can be utilized in privacy sensitive domains such as smart city applications, Industrial Internet of Things, Internet of Robotic Things, Internet of Medical Things, etc.