RobustPFL: Robust Personalized Federated Learning

Within the federated learning (FL) framework, the client collaboratively trains the model in coordination with a central server, while the training data can be kept locally on the client. Thus, the FL framework mitigates the privacy disclosure and costs related to conventional centralized machine learning. Nevertheless, current surveys indicate that FL still has problems in terms of communication efficiency and privacy risks. In this paper, to solve these problems, we develop an FL framework with communication-efficient and privacy-preserving (FLCP). To realize the FLCP, we design a novel compression algorithm with efficient communication, namely, adaptive weight compression FedAvg (AWC-FedAvg). On the basis of the non-independent and identically distributed (non-IID) and unbalanced data distribution in FL, a specific compression rate is provided for each client, and homomorphic encryption (HE) and differential privacy (DP) are integrated to provide demonstrable privacy protection and maintain the desirability of the model. Therefore, our proposed FLCP smoothly balances communication efficiency and privacy risks, and we prove its security against “honest-but-curious” servers and extreme collusion under the defined threat model. We evaluate the scheme by comparing it with state-of-the-art results on the MNIST and CIFAR-10 datasets. The results show that the FLCP performs better in terms of training efficiency and model accuracy than the baseline method.

Federated learning (FL) is a collaborative machine learning technique to train a global model (GM) without obtaining clients' private data. The main challenges in FL are statistical diversity among clients, limited computing capability among clients' equipment, and the excessive communication overhead between the server and clients. To address these challenges, we propose a novel sparse personalized FL scheme via maximizing correlation (FedMac). By incorporating an approximated l1 -norm and the correlation between client models and GM into standard FL loss function, the performance on statistical diversity data is improved and the communicational and computational loads required in the network are reduced compared with nonsparse FL. Convergence analysis shows that the sparse constraints in FedMac do not affect the convergence rate of the GM, and theoretical results show that FedMac can achieve good sparse personalization, which is better than the personalized methods based on the l2 -norm. Experimentally, we demonstrate the benefits of this sparse personalization architecture compared with the state-of-the-art personalization methods (e.g., FedMac, respectively, achieves 98.95%, 99.37%, 90.90%, 89.06%, and 73.52% accuracy on the MNIST, FMNIST, CIFAR-100, Synthetic, and CINIC-10 datasets under non-independent and identically distributed (i.i.d.) variants).

Federated learning (FL) enables the training of machine learning models on decentralized data, potentially improving data privacy. However, the FL distributed architecture is vulnerable to poisoning attacks. In this paper, we propose an FL method capable of mitigating these attacks through a triad of defense strategies: organizing clients into groups, evaluating the local performance of global models during training, and using a voting scheme during the inference phase. The proposed approach first divides the clients into randomly sampled groups, each generating a distinct global model. Each client trains a local model on their private data and submits it to the central server. The central server aggregates the local models within each group to generate the global models. Then, each client receives all global models, selects the best performing one as their new local model, and the process repeats until training is complete. During the inference phase, each client classifies its inputs according to a majority-based voting scheme among the global models. Our experiments using the HAR and MNIST datasets demonstrate that our method can effectively mitigate poisoning attacks without compromising the global model's performance.

Federated learning (FL) has become a promising technology to provide edge Artificial Intelligence (AI) due to its advantages in privacy protection and reduced communication costs. However, FL is still confronted with privacy leakage issues because the sharing local model may expose the training data information. Existing works typically utilize secure aggregation techniques to eliminate privacy leakage, where local model parameters in FL are obfuscated before they are sent to the aggregator. Nevertheless, secure aggregation makes poisoning attacks more convenient given that existing anomaly detection methods mostly require access to plaintext local models. A Poisoning-Defense Secure Aggregation in FL (PDSA-FL) is proposed to enhance the privacy protection of honest clients and defend against poisoning attacks from malicious clients. Specifically, a Secure Aggregation scheme based on Random Parameters Decomposition (SARPD) is designed to protect client privacy during the FL aggregation process and eliminates the impact of dropped clients on the aggregation results. Secondly, a Poisoning Detection method based on Similarity Grouping (PDSG) is proposed to mitigate the impact of poisoning attacks on the global model of FL without leaking client model parameters. The security analysis discusses the effectiveness of the proposed PDSA-FL in terms of privacy protection. Extensive simulation results show that PDSA-FL can effectively defend against poisoning attacks, significantly improve the convergence performance of global models, and reduce the computation time of clients.

In this paper, the convergence time of federated learning (FL), when deployed over a realistic wireless network, is studied. In particular, a wireless network is considered in which wireless users transmit their local FL models (trained using their locally collected data) to a base station (BS). The BS, acting as a central controller, generates a global FL model using the received local FL models and broadcasts it back to all users. Due to the limited number of resource blocks (RBs) in a wireless network, only a subset of users can be selected to transmit their local FL model parameters to the BS at each learning step. Moreover, since each user has unique training data samples, the BS prefers to include all local user FL models to generate a converged global FL model. Hence, the FL training loss and convergence time will be significantly affected by the user selection scheme. Therefore, it is necessary to design an appropriate user selection scheme that can select the users who can contribute toward improving the FL convergence speed more frequently. This joint learning, wireless resource allocation, and user selection problem is formulated as an optimization problem whose goal is to minimize the FL convergence time and the FL training loss. To solve this problem, a probabilistic user selection scheme is proposed such that the BS is connected to the users whose local FL models have significant effects on the global FL model with high probabilities. Given the user selection policy, the uplink RB allocation can be determined. To further reduce the FL convergence time, artificial neural networks (ANNs) are used to estimate the local FL models of the users that are not allocated any RBs for local FL model transmission at each given learning step, which enables the BS to improve the global model, the FL convergence speed, and the training loss. Simulation results show that the proposed approach can reduce the FL convergence time by up to 56% and improve the accuracy of identifying handwritten digits by up to 3%, compared to a standard FL algorithm.

Federated Learning (FL) in Consumer Internet of Things (CIoT) encounters significant privacy security threats when collaborative training Machine Learning models using data distributed among numerous smart CIoT devices. This paper’s objective is to evaluate the potential privacy risks associated with FL within CIoT context, specifically through the lens of passive malicious users. We propose an evaluation scheme (MIGAN) for privacy leakage, which incorporates various privacy inference attacks. First, this paper analyzes the characteristics of privacy inference attacks in FL and summarizes privacy security threats to guide the design of the MIGAN evaluation scheme. Then, to fully utilize the privacy information in FL, this paper proposes faster model inversion and white-box membership inference. Integrating these two modules into the MIGAN framework facilitates fast and high-quality privacy data reconstruction. Finally, this paper constructs the evaluation function to measure the correlation between compromised privacy data and users’ training data. Experiments conducted on the MNIST and CelebA datasets reveal that MIGAN’s data privacy reconstruction outperforms comparable methods. Notably, MIGAN quantifies the likelihood of potential privacy leakage at 14.7% for MNIST and 8.3% for CelebA in FL with 30 clients. These results validates the practicality and relevance of MIGAN in evaluating privacy leakage risks.

Utilizing the federated learning (FL) technique, data owners can collaboratively train artificial intelligence models, retaining all training data on their premises to minimize the potential for personal data breaches. However, self-interested users (e.g., free riders) bring new challenges that hinder the development of FL techniques. To this end, we propose a two-stage client selection scheme comprising a multiarmed bandit (MAB)-based candidate client selection method and an auction-based training client selection method. Specifically, our client selection scheme initially formulates the FL system into an MAB system, where clients are the arms and the server is the player. Then, we quantify the similarity between a local model and the server side, which is the designed metric for model aggregation and reward computation updating based on the fuzzy mathematical strategy. Next, based on the Thompson Sampling strategy, the server can intelligently determine the reward of each client, and clients with more significant rewards have the chance for local model training. With an auction method, the server can determine the training clients to reduce the training cost while maximizing each client’s revenue. Extensive experiments on real-world data sets demonstrate that the proposed scheme outperforms representative FL schemes (i.e., FedAvg, FedProx, FedMax, and MFL) regarding the model’s convergence rate and cost in FL systems with free riders.

Federated learning (FL) is known to be susceptible to model poisoning attacks in which malicious clients hamper the accuracy of the global model by sending manipulated model updates to the central server during the FL training process. Existing defenses mainly focus on Byzantine-robust FL aggregations, and largely ignore the impact of the underlying deep neural network (DNN) that is used to FL training. Inspired by recent findings on critical learning periods (CLP) in DNNs, where small gradient errors have irrecoverable impact on the final model accuracy, we propose a new defense, called a CLP-aware defense against poisoning of FL (DeFL). The key idea of DeFL is to measure fine-grained differences between DNN model updates via an easy-to-compute federated gradient norm vector (FGNV) metric. Using FGNV, DeFL simultaneously detects malicious clients and identifies CLP, which in turn is leveraged to guide the adaptive removal of detected malicious clients from aggregation. As a result, DeFL not only mitigates model poisoning attacks on the global model but also is robust to detection errors. Our extensive experiments on three benchmark datasets demonstrate that DeFL produces significant performance gain over conventional defenses against state-of-the-art model poisoning attacks.

In order to meet the extremely heterogeneous requirements of the next generation wireless communication networks, research community is increasingly dependent on using machine‐learning solutions for real‐time decision‐making and radio resource management. Traditional machine learning employs fully centralized architecture in which the entire training data is collected at one node for example, cloud server, that significantly increases the communication overheads and also raises severe privacy concerns. Toward this end, a distributed machine‐learning paradigm termed as federated learning (FL) has been proposed recently. In FL, each participating edge device trains its local model by using its own training data. Then, via the wireless channels the weights or parameters of the locally trained models are sent to the central parameter server (PS), that aggregates them and updates the global model. On one hand, FL plays an important role for optimizing the resources of wireless communication networks, on the other hand, wireless communications is crucial for FL. Thus, a “bidirectional” relationship exists between FL and wireless communications. Although FL is an emerging concept, many publications have already been published in the domain of FL and its applications for next generation wireless networks. Nevertheless, we noticed that none of the works have highlighted the bidirectional relationship between FL and wireless communications. Therefore, the purpose of this survey article is to bridge this gap in literature by providing a timely and comprehensive discussion on the interdependency between FL and wireless communications.

Federated learning is a distributed learning technique where machine learning models are trained on client devices in which the local training data resides. The training is coordinated via a central server which is, typically, controlled by the intended owner of the resulting model. By avoiding the need to transport the training data to the central server, federated learning improves privacy and efficiency. But it raises the risk of model theft by clients because the resulting model is available on every client device. Even if the application software used for local training may attempt to prevent direct access to the model, a malicious client may bypass any such restrictions by reverse engineering the application software. Watermarking is a well-known deterrence method against model theft by providing the means for model owners to demonstrate ownership of their models. Several recent deep neural network (DNN) watermarking techniques use backdooring: training the models with additional mislabeled data. Backdooring requires full access to the training data and control of the training process. This is feasible when a single party trains the model in a centralized manner, but not in a federated learning setting where the training process and training data are distributed among several client devices. In this paper, we present WAFFLE, the first approach to watermark DNN models trained using federated learning. It introduces a retraining step at the server after each aggregation of local models into the global model. We show that WAFFLE efficiently embeds a resilient watermark into models incurring only negligible degradation in test accuracy (-0.17%), and does not require access to training data. We also introduce a novel technique to generate the backdoor used as a watermark. It outperforms prior techniques, imposing no communication, and low computational (+3.2%) overhead¹¹The research report version of this paper is also available in https://arxiv.org/abs/2008.07298, and the code for reproducing our work can be found at https://github.com/ssg-research/WAFFLE.

Machine learning (ML) and Deep learning (DL) models are popular in many areas, from business, medicine, industries, healthcare, transportation, smart cities, and many more. However, the conventional centralized training techniques may not apply to upcoming distributed applications, which require high accuracy and quick response time. It is mainly due to limited storage and performance bottleneck problems on the centralized servers during the execution of various ML and DL-based models. However, federated learning (FL) is a developing approach to training ML models in a collaborative and distributed manner. It allows the full potential exploitation of these models with unlimited data and distributed computing power. In FL, edge computing devices collaborate to train a global model on their private data and computational power without sharing their private data on the network, thereby offering privacy preservation by default. But the distributed nature of FL faces various challenges related to data heterogeneity, client mobility, scalability, and seamless data aggregation. Moreover, the communication channels, clients, and central servers are also vulnerable to attacks which may give various security threats. Thus, a structured vulnerability and risk assessment are needed to deploy FL successfully in real-life scenarios. Furthermore, the scope of FL is expanding in terms of its application areas, with each area facing different threats. In this paper, we analyze various vulnerabilities present in the FL environment and design a literature survey of possible threats from the perspective of different application areas. Also, we review the most recent defensive algorithms and strategies used to guard against security and privacy threats in those areas. For a systematic coverage of the topic, we considered various applications under four main categories: space, air, ground, and underwater communications. We also compared the proposed methodologies regarding the underlying approach, base model, datasets, evaluation matrices, and achievements. Lastly, various approaches’ future directions and existing drawbacks are discussed in detail.

We study federated edge learning, where a global model is trained collaboratively using privacy-sensitive data at the edge of a wireless network. A parameter server (PS) keeps track of the global model and shares it with the wireless edge devices for training using their private local data. The devices then transmit their local model updates, which are used to update the global model, to the PS. The algorithm, which involves transmission over PS-to-device and device-to-PS links, continues until the convergence of the global model or lack of any participating devices. In this study, we consider device selection based on downlink channels over which the PS shares the global model with the devices. Performing digital downlink transmission, we design a partial device participation framework where a subset of the devices is selected for training at each iteration. Therefore, the participating devices can have a better estimate of the global model compared to the full device participation case which is due to the shared nature of the broadcast channel with the price of updating the global model with respect to a smaller set of data. At each iteration, the PS broadcasts different quantized global model updates to different participating devices based on the last global model estimates available at the devices. We investigate the best number of participating devices through experimental results for image classification using the MNIST dataset with biased distribution.

Federated Learning (FL) provides enhanced privacy over traditional centralized learning; unfortunately, it is also as susceptible to backdoor attacks, just like its centralized counterpart. Conventionally, in data poisoning-based backdoor attacks, all the malicious participants overlay the same single trigger pattern on a subset of their private data during local training. The same trigger is used to induce the backdoor in the otherwise benign global model at inference time. Such single trigger attacks can be detected and removed with relative ease as they undermine the distributed nature of FL. In this work, we focus on building an attack scheme where each batch of malicious clients uses sizably discrete local triggers during local training, with the ability to invoke the attack with a single small inference trigger during the global model testing. The larger size of the trigger pattern ensures prolonged attack longevity even after the termination of the attack. We conduct extensive experiments to show that our approach is far faster, stealthier, and more effective than the centralized trigger approach. The stealthiness of our work is explained using the DeepLIFT visual feature interpretation method.

Federated Learning (FL) is an effective framework for a distributed system that constructs a powerful global deep learning model, which diminishes the local bias and accommodates the successful aggregation of locally trained models with heterogeneous datasets. However, when local datasets have the non-IID attribute, the optimization metric tends to diverge or show unstable convergence in the trajectory space. This paper delves into building a global model for the distributed Smart Grid environment, with regionally cumulated three solar energy datasets from January 2017 to August 2021 in a decentralized power grid in South Korea via FL. This distributed energy network involves local properties and physical distance between the regions, which raises a fundamental question of “Will time-serially curated non-IID local features be effective in constructing a global regression model?”. This paper probes this question by leveraging FL and conducts the theoretically viable non-IID case-by-case convergence analysis, providing the interpretation of the embedded temporal non-IID features and application on real-world data. Moreover, most of the FL studies predetermine the global update period, which lacks applicability when adapting FL in actual practice. As FL is a cumulative-basis structure, the update term is a crucial factor that needs to be carefully selected. This paper articulates this problem and explores the effective update period via multiple experiments on the 4.5 years of solar energy dataset, and to the best of my knowledge, this is the first literature that presents the optimal update period in the FL regression in an energy domain.

Federated learning (FL) is gradually becoming a key learning paradigm in Privacy-preserving Machine Learning (ML) systems. In FL, a large number of clients cooperate with a central server to learn a shared model without sharing their own data sets. However, since there is a great disparity between the client data sets, standard FL is often hard to tune and suffers from performance degradation due to the inharmony among local models. To this end, in this paper we propose a novel FL scheme, termed client reputation federated learning (CRFL), which dynamically assesses the reputation of the clients participating in FL. Our method leverages techniques from model explanation, and aims at precisely measure each client's impact to the global model. To be specific, we first calculate the saliency-weighted variance on pixelwise relevance scores as the quality factor of a single sample. Then we extract activation function values at the last hidden layer to compute the divergence factor of individual data set. Finally, the server integrates these two factors as an assessment of the client reputation. By leveraging such assessment, CRFL can dynamically adjust the weights of the clients in each aggregation round, thus leading to a significant improvement over the baseline method in terms of model accuracy and convergence rate. Intensive experiments are conducted on the MNIST and CIFAR-10 data sets, and experimental results demonstrate the efficacy of the proposed method.