Wirelessly Powered Federated Learning Networks: Joint Power Transfer, Data Sensing, Model Training, and Resource Allocation

Federated Learning (FL) is very appealing for its privacy benefits: essentially, a global model is trained with updates computed on mobile devices while keeping the data of users local. Standard FL infrastructures are however designed to have no energy or performance impact on mobile devices, and are therefore not suitable for applications that require frequent (online) model updates, such as news recommenders. This paper presents FLeet, the first Online FL system, acting as a middleware between the Android OS and the machine learning application. FLeet combines the privacy of Standard FL with the precision of online learning thanks to two core components: (i) I-Prof, a new lightweight profiler that predicts and controls the impact of learning tasks on mobile devices, and (ii) AdaSGD, a new adaptive learning algorithm that is resilient to delayed updates. Our extensive evaluation shows that Online FL, as implemented by FLeet, can deliver a 2.3x quality boost compared to Standard FL, while only consuming 0.036% of the battery per day. I-Prof can accurately control the impact of learning tasks by improving the prediction accuracy up to 3.6x (computation time) and up to 19x (energy). AdaSGD outperforms alternative FL approaches by 18.4% in terms of convergence speed on heterogeneous data.

With the growth of participating clients, the centralized parameter server (PS) will seriously limit the scale and efficiency of Federated Learning (FL). A straightforward approach to scale up the FL system is to construct a Parallel FL (PFL) system with multiple parallel PSes. However, it is unclear whether PFL can really accelerate FL or reduce the training time of FL. Even if the answer is yes, it is non-trivial to design a highly efficient parameter average algorithm for a PFL system. In this paper, we propose a completely parallelizable FL algorithm called P-FedAvg under the PFL architecture. P-FedAvg extends the well-known FedAvg algorithm by allowing multiple PSes to cooperate and train a learning model together. In P-FedAvg, each PS is only responsible for a fraction of total clients, but PSes can mix model parameters in a dedicatedly designed way so that the FL model can well converge. Different from heuristic-based algorithms, P-FedAvg is with theoretical guarantees. To be rigorous, we theoretically analyze the convergence rate of P-FedAvg in terms of the number of conducted iterations, the communication cost of each global iteration and the optimal weights for each PS to mix parameters with its neighbors. Based on theoretical analysis, we conduct a case study on five typical overlay topolgoies formed by PSes to further examine the communication efficiency under different topologies, and investigate how the overlay topology affects the convergence rate, communication cost and robustness of a PFL system. Lastly, we perform extensive experiments with real datasets to verify our analysis and demonstrate that P-FedAvg can significantly speed up FL than traditional FedAvg and other competitive baselines. We believe that our work can help to lay a theoretical foundation for building more efficient PFL systems.

Mobile crowdsensing (MCS) counting on the mobility of massive workers helps the requestor accomplish various sensing tasks with more flexibility and lower cost. However, for the conventional MCS, the large consumption of communication resources for raw data transmission and high requirements on data storage and computing capability hinder potential requestors with limited resources from using MCS. To facilitate the widespread application of MCS, we propose a novel MCS learning framework leveraging on blockchain technology and the new concept of edge intelligence based on federated learning (FL), which involves four major entities, including requestors, blockchain, edge servers and mobile devices as workers. Even though there exist several studies on blockchain-based MCS and blockchain-based FL, they cannot solve the essential challenges of MCS with respect to accommodating resource-constrained requestors or deal with the privacy concerns brought by the involvement of requestors and workers in the learning process. To fill the gaps, four main procedures, i.e., task publication, data sensing and submission, learning to return final results, and payment settlement and allocation, are designed to address major challenges brought by both internal and external threats, such as malicious edge servers and dishonest requestors. Specifically, a mechanism design based data submission rule is proposed to guarantee the data privacy of mobile devices being truthfully preserved at edge servers; consortium blockchain based FL is elaborated to secure the distributed learning process; and a cooperation-enforcing control strategy is devised to elicit full payment from the requestor. Extensive simulations are carried out to evaluate the performance of our designed schemes.

Privacy and data security have become the new hot topic for regulators in recent years. As a result, Federated Learning (FL) (also called collaborative learning) has emerged as a new training paradigm that allows multiple, geographically distributed nodes to learn a Deep Learning (DL) model together without sharing their data. Blockchain is becoming a new trend as data protection and privacy are concerns in many sectors. Technology is leading the world and transforming into a global village where everything is accessible and transparent. We have presented a blockchain enabled security model using FL that can generate an enhanced DL model without sharing data and improve privacy through higher security and access rights to data. However, existing FL approaches also have unique security vulnerabilities that malicious actors can exploit and compromise the trained model. The FL method is compared to the other known approaches. Users are more likely to choose the latter option, i.e., providing local but private data to the server and using ML apps, performing ML operations on the devices without benefiting from other users’ data, and preventing direct access to raw data and local training of ML models. FL protects data privacy and reduces data transfer overhead by storing raw data on devices and combining locally computed model updates. We have investigated the feasibility of data and model poisoning attacks under a blockchain-enabled FL system built alongside the Ethereum network and the traditional FL system (without blockchain). This work fills a knowledge gap by proposing a transparent incentive mechanism that can encourage good behavior among participating decentralized nodes and avoid common problems and provides knowledge for the FL security literature by investigating current FL systems.

Blockchain network (BCN)-based Self-Sovereign Identity (SSI) has emerged lately as an identity and access management framework that is based on Distributed Ledger Technology (DLT) and allows users to control their own data. Federated Learning (FL), on the other hand, provides a collaborative framework to update Machine Learning (ML) models without relying explicitly on data exchange between the users. This paper investigates identity management and authentication for vehicle users in the context of FL. We propose a novel approach based on blockchain-based SSI, which focuses on maintaining the authenticity and integrity of vehicle users’ identities and data exchanged between the users and the aggregation server during the execution of the FL iterations. A primary objective of this paper is to achieve shorter durations for credential operations in an FL setting as the system size scales out. Integrating BCN-based SSI into the FL framework addresses several critical FL challenges, ensuring enhanced system security and operational integrity. This synergy of BCN-based SSI with federated learning enables robust identity verification providing a solution to fundamental trustworthiness issues in FL without sacrificing the benefits of decentralized data control, improving both the performance and reliability of the FL system. Experimental results suggest that the proposed FL-based system, together with credential management on a blockchain platform, has the potential to significantly improve data integrity and ensure the authentication of users. More specifically, the results of the FL system demonstrate that it takes longer (on the order of a hundred seconds) as the number of rounds and clients increase, while the implemented Decentralized Identifier (DID) system relying on BCN-based SSI has dramatically shorter dedicated time for completing credential operations.

Federated Learning (FL) is considered the key, enabling approach for privacy-preserving, distributed machine learning (ML) systems. FL requires the periodic transmission of ML models from users to the server. Therefore, communication via resource-constrained networks is currently a fundamental bottleneck in FL, which is restricting the ML model complexity and user participation. One of the notable trends to reduce the communication cost of FL systems is gradient compression, in which techniques in the form of sparsification are utilized. However, these methods utilize a single compression rate for all users and do not consider communication heterogeneity in a real-world FL system. Therefore, these methods are bottlenecked by the worst communication capacity across users. Further, sparsification methods are non-adaptive and do not utilize the redundant, similar information across users' ML models for compression. In this paper, we introduce a novel Dynamic Sparsification for Federated Learning (DSFL) approach that enables users to compress their local models based on their communication capacity at each iteration by using two novel sparsification methods: layer-wise similarity sparsification (LSS) and extended top- <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$K$</tex> sparsification. LSS enables DSFL to utilize the global redundant information in users' models by using the Centralized Kernel Alignment (CKA) similarity for sparsification. The extended top- <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$K$</tex> model sparsification method empowers DSFL to accommodate the heterogeneous communication capacity of user devices by allowing different values of sparsification rate <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$K$</tex> for each user at each iteration. Our extensive experimental results <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> All code and experiments are publicly available at: https://github.com/mahdibeit/DSFL. on three datasets show that DSFL has a faster convergence rate than fixed sparsification, and as the communication heterogeneity increases, this gap increases. Further, our thorough experimental investigations uncover the similarities of user models across the FL system.

Privacy-preserving Federated Learning (FL) based on Differential Privacy (DP) protects clients’ data by adding DP noise to samples’ gradients and has emerged as a de facto standard for data privacy in FL. However, the accuracy of global models in DP-based FL may be reduced significantly when rogue clients occur who deviate from the preset DP-based FL approaches and selfishly inject excessive DP noise beyond expectations, thereby applying a smaller privacy budget in the DP mechanism to ensure a higher level of security. Existing DP-based FL fails to prevent such attacks as they are imperceptible. Under the DP-based FL system and random Gaussian noise, the local model parameters of the rogue clients and the honest clients have identical distributions. In particular, the rogue local models show a low performance, but directly filtering out lower-performance local models compromises the generalizability of global models, as local models trained on scarce data also behave with low performance in the early epoch. In this paper, we propose ReFL, a novel privacy-preserving FL system that enforces DP and avoids the accuracy reduction of global models caused by excessive DP noise of rogue clients. Based on the observation that rogue local models with excessive DP noise and honest local models trained on scarce data have different performance patterns in long-term training epochs, we propose a long-term contribution incentives scheme to evaluate clients’ reputations and identify rogue clients. Furthermore, we design a reputation-based aggregation to avoid the damage of rogue clients’ models on the global model accuracy, based on the incentive reputation. Extensive experiments demonstrate ReFL guarantees the global model accuracy performance 0.77% - 81.71% higher than existing DP-based FL methods in the presence of rogue clients.

Edge servers and mobile devices are often assigned a large number of computing tasks. However, the data involved in computing tasks is often sensitive in terms of privacy. Our initial proposal is a federated edge learning strategy based on real-world scenarios, which combines blurred data or pseudo shared data. Federated learning is used to train device models with the aim of protecting privacy while enabling mobile devices to more effectively utilize data for decision-making. In the case of limited energy on mobile devices, we propose a federated edge learning algorithm with blurred data sharing. This algorithm can generate more accurate models by uploading partially blurred data. In order to further improve model accuracy and protect privacy of mobile devices, we propose a federated edge learning algorithm with pseudo data sharing based on dataset distillation and generative adversarial networks (GANs) in scenarios with relatively sufficient energy. The experimental results on several traditional datasets show that our proposed algorithms outperform traditional algorithms in terms of accuracy and energy consumption.

This paper studies a federated learning (FL) system, where <i>multiple</i> FL services co-exist in a wireless network and share common wireless resources. It fills the void of wireless resource allocation for multiple simultaneous FL services in the existing literature. Our method designs a two-level resource allocation framework comprising <i>intra-service</i> resource allocation and <i>inter-service</i> resource allocation. The intra-service resource allocation problem aims to minimize the length of FL rounds by optimizing the bandwidth allocation among the clients of each FL service. Based on this, an inter-service resource allocation problem is further considered, which distributes bandwidth resources among multiple simultaneous FL services. We consider both cooperative and selfish providers of the FL services. For cooperative FL service providers, we design a distributed bandwidth allocation algorithm to optimize the overall performance of multiple FL services, meanwhile catering it to the fairness among FL services and the privacy of clients. For selfish FL service providers, a new auction scheme is designed with the FL service providers as the bidders and the network operator as the auctioneer. The designed auction scheme strikes a balance between the overall FL performance and fairness. Our simulation results show that the proposed algorithms outperform other benchmarks under various network conditions.

Federated learning (FL) enables the training of a shared collaborative machine learning model while keeping all the confidential training data on distributed devices. The FL state-of-the-art considers a monopolist FL task publisher. However, we present a FL marketplace where multiple FL task publishers and mobile devices co-exist for a set of diverse and varying learning tasks. Mobile devices participating in the training of FL models provides pay-as-you-go (i.e. using blockchain-based cryptocurrencies) FL training services to the FL task publishers. In the proposed framework, multiple FL task publishers may compete with each other and the participating workers (i.e. mobile devices) can choose one FL task publisher over another for participation in the training of a global model. We utilize code offloading for enabling customized FL pipelines in mobile devices and mitigating the model heterogeneity inherent in varying and changing FL tasks published by the task publishers. Experimental results indicate the efficacy of the proposed framework.

The heterogeneous connections in metaverse environments pose vulnerabilities to cyber-attacks. To prevent and mitigate malicious network activities in a distributed metaverse, conventional intrusion detection systems (IDS) have communication overhead and privacy concerns. Federated learning (FL) techniques are widely employed to develop IDS frameworks and enable privacy-preserving collaborative learning schemes in decentralized ecosystems. However, the vanilla FL system utilizes a centralized FL aggregation technique, which introduces a single point of failure (SPoF) and potential unauthorized aggregators, allowing malicious clients to inject false data parameters, known as poisoning attacks. Furthermore, low-quality clients in the FL system can result in degraded model performance and hinder convergence. This study proposes a secure and reliable blockchain-aided federated learning (BFL)-based IDS framework using a lightweight model for securing metaverse. An authorized federated IDS is proposed to establish a trustworthy decentralized aggregation mechanism, utilizing proof-of-authority (PoA) consensus. The proposed federated IDS implemented a hybrid client selection (HCS) technique, considering the accuracy and reputation of client histories, to select high-quality metaverse edge devices. Additionally, a fairness ERC-20 token-based incentive mechanism was developed to reward selected FL clients as a token of appreciation for their contribution to the FL training processes. According to the IDS framework measurements, the proposed model performs better than the existing approaches for detecting cyber-attacks in metaverse environments, achieving an accuracy of 99.28% with trainable parameters of 1.8K and mega floating-point operations (MFLOPs) of 0.0016.

Federated learning (FL) is an emerging machine learning paradigm. With FL, distributed data owners aggregate their model updates to train a shared deep neural network collaboratively, while keeping the training data locally. However, FL has little control over the local data and the training process. Therefore, it is susceptible to poisoning attacks, in which malicious or compromised clients use malicious training data or local updates as the attack vector to poison the trained global model. Moreover, the performance of existing detection and defense mechanisms drops significantly in a scaled-up FL system with non-iid data distributions. In this paper, we propose a defense scheme named CONTRA to defend against poisoning attacks, e.g., label-flipping and backdoor attacks, in FL systems. CONTRA implements a cosine-similarity-based measure to determine the credibility of local model parameters in each round and a reputation scheme to dynamically promote or penalize individual clients based on their per-round and historical contributions to the global model. With extensive experiments, we show that CONTRA significantly reduces the attack success rate while achieving high accuracy with the global model. Compared with a state-of-the-art (SOTA) defense, CONTRA reduces the attack success rate by 70% and reduces the global model performance degradation by 50%.

Utilizing the federated learning (FL) technique, data owners can collaboratively train artificial intelligence models, retaining all training data on their premises to minimize the potential for personal data breaches. However, self-interested users (e.g., free riders) bring new challenges that hinder the development of FL techniques. To this end, we propose a two-stage client selection scheme comprising a multiarmed bandit (MAB)-based candidate client selection method and an auction-based training client selection method. Specifically, our client selection scheme initially formulates the FL system into an MAB system, where clients are the arms and the server is the player. Then, we quantify the similarity between a local model and the server side, which is the designed metric for model aggregation and reward computation updating based on the fuzzy mathematical strategy. Next, based on the Thompson Sampling strategy, the server can intelligently determine the reward of each client, and clients with more significant rewards have the chance for local model training. With an auction method, the server can determine the training clients to reduce the training cost while maximizing each client’s revenue. Extensive experiments on real-world data sets demonstrate that the proposed scheme outperforms representative FL schemes (i.e., FedAvg, FedProx, FedMax, and MFL) regarding the model’s convergence rate and cost in FL systems with free riders.

In this paper, we consider a micro unmanned aerial vehicle (UAV) swarm network enabled by simultaneous wireless information and power transfer (SWIPT). The network consists of one base station (BS) and multiple micro UAVs, where the BS aims to train a machine learning model over the data that resides on each UAV by utilizing federated learning (FL). Particularly, at each iteration of FL, the BS broadcasts both the global model and energy simultaneously to all UAVs, and each UAV relies on its harvested and battery-stored energy to train the received model and then upload it to the BS for global model aggregation. To improve the learning performance, we formulate a problem of maximizing the percentage of scheduled UAVs at each iteration of FL by jointly optimizing UAV scheduling and sub-channel assignment, as well as broadcasting/uploading time and power allocation. The formulated problem is a challenging mixed integer nonlinear programming problem and is NP-hard in general. By exploiting the special property of the problem structure, we develop a low-complexity suboptimal algorithm to efficiently solve the formulated problem. Numerical results show the advantages of our suboptimal algorithm.

Federated Learning (FL) is a paradigm in distributed machine learning, which has gained significant attention in the recent years especially in the domain of Internet of Things. Federated Learning saves communication bandwidth when compared to centralized machine learning process and is also considered to be privacy preserving as the raw data at the clients need not be transmitted to the server for the FL learning. Recent studies on FL have exposed privacy vulnerabilities particularly in the form of Gradient Leakage Attacks (GLA) in which the adversaries can generate training data of the client from shared gradients or model updates from clients. The available solutions in the literature for GLA based on perturbing the gradients from clients leads to a drop in the performance of the FL system while attempting to preserve privacy. We propose HAFedL, an improved novel hessian aware adaptive privacy preserving FL scheme in which the performance of the model is not significantly affected due to the aspects introduced in the FL architecture to improve the privacy of the model against the GLA. The HAFedL is also robust to the data heterogeneity and device heterogeneity (particularly straggler effect) which may be present in the clients participating in the FL. The performance of HAFedL is tested for two applications- IoT device identification and digit classification. The proposed HAFedL scheme can be utilized in privacy sensitive domains such as smart city applications, Industrial Internet of Things, Internet of Robotic Things, Internet of Medical Things, etc.