Bandwidth Allocation for Multiple Federated Learning Services in Wireless Edge Networks

Federated learning (FL) has been vigorously promoted in wireless edge networks as it fosters collaborative training of machine learning (ML) models while preserving individual user privacy and data security. In conventional FL, user equipment (UE) and an aggregator can collaboratively train shared global ML models by transmitting interactive ML models. In wireless edge networks, heterogeneity of multi-dimensional resources (e.g., computing and communication resources) used to train and transmit FL models may introduce stragglers, characterized by a slow update and/or transmission of local models. The stragglers can significantly degrade learning performance of FL, as the slowest participating UE can dramatically slow down the entire convergence. In this paper, to alleviate the negative impact of stragglers, we propose a dynamic straggler-aware clustering based FL (FeDSC) mechanism via adaptive UE clustering. Specifically, we first group participating UEs into multiple clusters based on their available computing and wireless resources. Then, we propose an adaptive clustering scheme to synchronously update the cluster aggregation model. Meanwhile, an edge server performs global aggregation of different cluster models in an asynchronous manner. Finally, we theoretically demonstrate the convergence of our proposed mechanism via numerical results. Numerical results show that our proposed mechanism can effectively reduce training time and wireless bandwidth consumption, while improving training efficiency and guaranteeing learning accuracy.

In this paper, the optimization of network performance to support the deployment of federated learning (FL) is investigated. In particular, in the considered model, each user owns a machine learning (ML) model by training through its own dataset, and then transmits its ML parameters to a base station (BS) which aggregates the ML parameters to obtain a global ML model and transmits it to each user. Due to limited radio frequency (RF) resources, the number of users that participate in FL is restricted. Meanwhile, each user uploading and downloading the FL parameters may increase communication costs thus reducing the number of participating users. To this end, we propose to introduce visible light communication (VLC) as a supplement to RF and use compression methods to reduce the resources needed to transmit FL parameters over wireless links so as to further improve the communication efficiency and simultaneously optimize wireless network through user selection and resource allocation. This user selection and bandwidth allocation problem is formulated as an optimization problem whose goal is to minimize the training loss of FL. We first use a model compression method to reduce the size of FL model parameters that are transmitted over wireless links. Then, the optimization problem is separated into two subproblems. The first subproblem is a user selection problem with a given bandwidth allocation, which is solved by a traversal algorithm. The second subproblem is a bandwidth allocation problem with a given user selection, which is solved by a numerical method. The ultimate user selection and bandwidth allocation are obtained by iteratively compressing the model and solving these two subproblems. Simulation results show that the proposed FL algorithm can improve the accuracy of object recognition by up to 16.7% and improve the number of selected users by up to 68.7%, compared to a conventional FL algorithm using only RF.

Federated learning (FL) has found many successes in wireless communications; however, the implementation of FL has been hindered by the energy limitation of mobile devices (MDs) and the availability of training data at MDs. Wireless power transfer (WPT) and mobile crowdsensing (MCS) are promising technologies that can be leveraged to power energy-limited MDs and acquire data for learning tasks. How to integrate WPT and MCS towards sustainable FL solutions is a research topic entirely missing from the open literature. This work for the first time investigates a resource allocation problem in collaborative sensing-assisted sustainable FL (S2FL) networks with the goal of minimizing the total completion time. In particular, we investigate a practical harvesting-sensing-training-transmitting protocol in which energy-limited MDs first harvest energy from RF signals, use it to gain a reward for user participation, sense the training data from the environment, train the local models at MDs, and transmit the model updates to the edge server. The total completion time minimization problem of jointly optimizing power transfer, transmit power allocation, data sensing, bandwidth allocation, local model training, and data transmission is complicated due to the non-convex objective function, highly non-convex constraints, and strongly coupled variables. In order to solve that problem, we apply the decomposition technique and develop a computationally-efficient path-following algorithm to obtain the solution. In particular, inner convex approximations are developed for the resource allocation subproblem, and the subproblems are performed alternatively in an iterative fashion. Simulation results are provided to evaluate the effectiveness of the proposed S2FL algorithm in reducing the completion time up to 21.45% in comparison with other benchmark schemes. Further, we investigate an extension of our work from frequency division multiple access (FDMA) to non-orthogonal multiple access (NOMA) and show that NOMA can speed up the total completion time 8.36% on average of the considered FL system.

In this paper, the convergence time of federated learning (FL), when deployed over a realistic wireless network, is studied. In particular, a wireless network is considered in which wireless users transmit their local FL models (trained using their locally collected data) to a base station (BS). The BS, acting as a central controller, generates a global FL model using the received local FL models and broadcasts it back to all users. Due to the limited number of resource blocks (RBs) in a wireless network, only a subset of users can be selected to transmit their local FL model parameters to the BS at each learning step. Moreover, since each user has unique training data samples, the BS prefers to include all local user FL models to generate a converged global FL model. Hence, the FL training loss and convergence time will be significantly affected by the user selection scheme. Therefore, it is necessary to design an appropriate user selection scheme that can select the users who can contribute toward improving the FL convergence speed more frequently. This joint learning, wireless resource allocation, and user selection problem is formulated as an optimization problem whose goal is to minimize the FL convergence time and the FL training loss. To solve this problem, a probabilistic user selection scheme is proposed such that the BS is connected to the users whose local FL models have significant effects on the global FL model with high probabilities. Given the user selection policy, the uplink RB allocation can be determined. To further reduce the FL convergence time, artificial neural networks (ANNs) are used to estimate the local FL models of the users that are not allocated any RBs for local FL model transmission at each given learning step, which enables the BS to improve the global model, the FL convergence speed, and the training loss. Simulation results show that the proposed approach can reduce the FL convergence time by up to 56% and improve the accuracy of identifying handwritten digits by up to 3%, compared to a standard FL algorithm.

Federated Learning (FL) is an emerging technique to enhance edge intelligence, where mobile devices train machine learning models collaboratively with their local data. Limited energy on devices and scarce wireless bandwidth can notably impact the convergence of FL over wireless networks, and thus device scheduling and resource allocation are critical. In this paper, we propose a joint device scheduling and resource allocation scheme to maximize the model accuracy under total training delay and device energy budgets. Since FL consists of multiple training rounds, there is an inherent trade-off between per-round delay, per-round energy consumption, and the total number of rounds. To find solution, we decouple the accuracy maximization problem into two sub-problems. First, given a scheduling policy, the bandwidth allocation and local computing frequency are jointly optimized to maximize the number of rounds that can be conducted. Then, a device scheduling policy is proposed to balance the trade-off between the per-round energy and delay cost and the number of rounds, with the ultimate goal of accuracy optimization. Experiments on various learning tasks and datasets show that the proposed scheme can greatly improve the convergence rate of resource-constrained FL.

Mobile edge computing aims to deploy mobile applications at the edge of wireless networks. Federated learning in mobile edge computing is a forward-looking distributed framework for deploying deep learning algorithms in many application scenarios. One challenge of federated learning in mobile edge computing is data heterogeneity since the unified model of federated learning performs poorly when client data are non-independent and identically distributed. Personalized federated learning can obtain amazing models in scenarios where client data are non-independent and identically distributed. This is because the personalized model captures the features of users’ local data more accurately than the unified model. The personalized federated learning problem under two-tier (server-client) federated learning structures has been widely studied and applied. However, a lot of research results exhibit three distinct limitations: 1) suboptimal communication efficiency, 2) slow model convergence, and 3) underutilization of the relationships within user data, resulting in lower accuracy of personalized models. In this paper, we present the first personalized federated learning algorithm based on the client-edge-cloud structure. The edge server is responsible for model personalization and employs a learnable mixing parameter to mix the local model and the global model. We also utilize two learnable normalization parameters trained by clients to improve the performance of personalized models. Furthermore, in order to facilitate the collaboration among edge servers, we propose a similarity aggregation method to assign aggregation weights based on the Tanimoto coefficients between models. The experimental results show that the proposed algorithm not only increases the convergence speed of personalized models but also improves their testing accuracy.

Federated learning (FL) is widely used in privacy sensitive applications for isolated data islands, with the aim of achieving distributed model training, privacy enhancement and model sharing. Electromyographic (EMG) signals are a type of data collected from wearable sensors of subjects which are distributed on multiple devices, highly personalized and play an important role in several applications including prosthetic hand control, sign languages, grasp recognition, etc. This paper utilizes the FL method to detect single and combined finger movements based on EMG signals. The existing research on FL for wearable healthcare faces challenges of variable probability distributions of data, the need for prerequisite knowledge of server model and computational burdens in parameter transmission. To address these problems, this paper proposes a communication efficient FL framework in which each device only needs to transmit the weight matrices of local models to the server for model aggregation. To further reduce the FL transmission delay, a joint learning and resource allocation problem is formulated via optimizing transmit power of each device, time allocation, and user selection. To solve the delay minimization problem, the objective function is first converted to a tractable expression and then the difference of two convex functions programming is adopted. Simulation results using real EMG signals show that the proposed FL framework with personalized training process successfully detects single and combined finger movements for distributed users. Two public EMG datasets with 10 and 15 different finger movements are employed. Over 98% overall test accuracy is achieved in both datasets which surpasses the conventional learning framework by 1.6% and 0.5% on average. Different scenarios with regard to access points and users are investigated and the convexity of the proposed model is discussed.

Due to the advanced capabilities of the Internet of Vehicles (IoV) components such as vehicles, Roadside Units (RSUs) and smart devices as well as the increasing amount of data generated, Federated Learning (FL) becomes a promising tool given that it enables privacy-preserving machine learning that can be implemented in the IoV. However, the performance of the FL suffers from the failure of communication links and missing nodes, especially when continuous exchanges of model parameters are required. Therefore, we propose the use of Unmanned Aerial Vehicles (UAVs) as wireless relays to facilitate the communications between the IoV components and the FL server and thus improving the accuracy of the FL. However, a single UAV may not have sufficient resources to provide services for all iterations of the FL process. In this paper, we present a joint auction-coalition formation framework to solve the allocation of UAV coalitions to groups of IoV components. Specifically, the coalition formation game is formulated to maximize the sum of individual profits of the UAVs. The joint auction-coalition formation algorithm is proposed to achieve a stable partition of UAV coalitions in which an auction scheme is applied to solve the allocation of UAV coalitions. The auction scheme is designed to take into account the preferences of IoV components over heterogeneous UAVs. The simulation results show that the grand coalition, where all UAVs join a single coalition, is not always stable due to the profit-maximizing behavior of the UAVs. In addition, we show that as the cooperation cost of the UAVs increases, the UAVs prefer to support the IoV components independently and not to form any coalition.

These days, with the rising computational capabilities of wireless user equipment such as smartphones, tablets, and vehicles, along with growing concerns about sharing private data, a novel machine learning model called federated learning (FL) has emerged. FL enables the separation of data acquisition and computation at the central unit, which is different from centralized learning that occurs in a data center. FL is typically used in a wireless edge network where communication resources are limited and unreliable. Bandwidth constraints necessitate scheduling only a subset of UEs for updates in each iteration, and because the wireless medium is shared, transmissions are susceptible to interference and are not assured. The article discusses the significance of Machine Learning in wireless communication and highlights Federated Learning (FL) as a novel approach that could play a vital role in future mobile networks, particularly 6G and beyond.

A game incentive mechanism for energy efficient federated learning in computing power networks

With advances in the edge computing (EC) and federated learning (FL) technologies in jointcloud, the edge FL service market has emerged recently and it requires trading edge resources between model requesters and data owners to complete FL tasks, which needs to incentivize sufficient data owners to participate in model training tasks. However, the limitations of resource trading and incentive design for edge FL service market have not been well addressed. In this paper, we propose a two-level blockchain-aided resource trading mechanism for encouraging appropriate edge servers to compete for dynamic FL tasks from the market while incentivizing data owners to participate in the FL tasks. At the upper level, we apply the deep learning-based reverse auction to model the dynamics of the task server selection process, with the aim of maximizing the total social welfare of the edge FL service market, where the edge server, as a seller, considers not only the data contribution of edge devices but also the cost of using blockchain when bidding. At the lower level, the edge servers offer rewards in exchange for the data owners’ participation, while the parameter aggregation is completed through the blockchain in a decentralized manner, which improves the FL’s robustness. Then, we utilize the Stackelberg game to model the dynamic process that the data owners compete for the servers’ revenue. We conduct extensive simulation experiments and the experimental results show that the proposed mechanism is able to get maximized social welfare and provide effective insights and strategies for the resource trading in the edge FL market to complete the federated training.

With the growing number of Federated Learning (FL) applications in smart homes, it becomes crucial to manage communication and computation resources within the smart home so that FL applications can complete their training on time. While computation offloading has relieved the challenge of timely completion of applications in case of high competition for local resources, privacy of the smart home data remains a critical concern. This paper introduces ERAFL, a resource allocation and computation offloading algorithm running on a home gateway. Unlike privacy-oblivious prior works, ERAFL considers privacy-sensitivity level of FL training data in offloading decision, prioritizing local processing of more sensitive data, e.g., biological personal data. Moreover, in case of insufficient local resources, ERAFL offloads a part of data and accelerates training by leveraging parallel training on the cloud and the edge device. It also imposes limits on the amount of offloaded data or performs the training either locally or remotely to ensure model accuracy. Our simulation results show that ERAFL can satisfy more FL training tasks and reduce data privacy leakage in comparison to the baselines that do not consider partial offloading, privacy sensitivity of application data or resource allocation.

In order to meet the extremely heterogeneous requirements of the next generation wireless communication networks, research community is increasingly dependent on using machine‐learning solutions for real‐time decision‐making and radio resource management. Traditional machine learning employs fully centralized architecture in which the entire training data is collected at one node for example, cloud server, that significantly increases the communication overheads and also raises severe privacy concerns. Toward this end, a distributed machine‐learning paradigm termed as federated learning (FL) has been proposed recently. In FL, each participating edge device trains its local model by using its own training data. Then, via the wireless channels the weights or parameters of the locally trained models are sent to the central parameter server (PS), that aggregates them and updates the global model. On one hand, FL plays an important role for optimizing the resources of wireless communication networks, on the other hand, wireless communications is crucial for FL. Thus, a “bidirectional” relationship exists between FL and wireless communications. Although FL is an emerging concept, many publications have already been published in the domain of FL and its applications for next generation wireless networks. Nevertheless, we noticed that none of the works have highlighted the bidirectional relationship between FL and wireless communications. Therefore, the purpose of this survey article is to bridge this gap in literature by providing a timely and comprehensive discussion on the interdependency between FL and wireless communications.

As the Augmented Intelligence of Things (AIoT) advances within vehicle-road coordination systems, challenges related to road traffic data transmission and processing are being increasingly addressed. However, this progress also brings significant risks of privacy data leakage. Federated learning (FL), a distributed machine learning paradigm, effectively safeguards client data privacy by allowing multiple participants to collaboratively train models while keeping their data localized. Despite its benefits, FL faces challenges, such as model parameter leakage and Byzantine attacks. To tackle these issues, this article introduces an anonymous and efficient FL framework for vehicle-road coordination systems (AEFL), designed to ensure a secure and reliable vehicle data transmission process. This architecture incorporates a novel group pairing onion routing protocol, which leverages pairing cryptography principles for hierarchical data encryption. During the routing process, relay group nodes decrypt the corresponding layer, ensuring both data confidentiality and node anonymity. Additionally, a sampling method is proposed to accurately identify Byzantine vehicle nodes, enhancing the precision of FL aggregation without compromising overall model performance. Experimental results show that AEFL outperforms the classic TOR anonymous routing protocol, achieving a 100% message delivery rate more quickly. Under the same conditions, the anonymity of the source node and the destination node improves by 3.9% and 1.9%, respectively. When half of the nodes are compromised, path anonymity can be increased by 24.8%. Furthermore, our framework excels in FL aggregation efficiency, with a Byzantine adversary detection accuracy of up to 99%.

Federated learning (FL) is a privacy-preserving distributed learning paradigm that enables clients to jointly train a global model. In real-world FL implementations, client data could have label noise, and different clients could have vastly different label noise levels. Although there exist methods in centralized learning for tackling label noise, such methods do not perform well on heterogeneous label noise in FL settings, due to the typically smaller sizes of client datasets and data privacy requirements in FL. In this paper, we propose FedCorr, a general multi-stage framework to tackle heterogeneous label noise in FL, without making any assumptions on the noise models of local clients, while still maintaining client data privacy. In particular, (1) FedCorr dynamically identifies noisy clients by exploiting the dimensionalities of the model prediction subspaces independently measured on all clients, and then identifies incorrect labels on noisy clients based on persample losses. To deal with data heterogeneity and to increase training stability, we propose an adaptive local proximal regularization term that is based on estimated local noise levels. (2) We further finetune the global model on identified clean clients and correct the noisy labels for the remaining noisy clients after finetuning. (3) Finally, we apply the usual training on all clients to make full use of all local data. Experiments conducted on CIFAR-10/100 with federated synthetic label noise, and on a real-world noisy dataset, Clothing1M, demonstrate that FedCorr is robust to label noise and substantially outperforms the state-of-the-art methods at multiple noise levels.