Federated Learning in Vehicular Networks

Many of the machine learning tasks are focused on centralized learning (CL), which requires the transmission of local datasets from the clients to a parameter server (PS) leading to a huge communication overhead. Federated learning (FL) overcomes this issue by allowing the clients to send only the model updates to the PS instead of the whole dataset. In this way, FL brings the learning to edge level, wherein powerful computational resources are required on the client side. This requirement may not always be satisfied because of diverse computational capabilities of edge devices. We address this through a novel hybrid federated and centralized learning (HFCL) framework to effectively train a learning model by exploiting the computational capability of the clients. In HFCL, only the clients who have sufficient resources employ FL; the remaining clients resort to CL by transmitting their local dataset to PS. This allows all the clients to collaborate on the learning process regardless of their computational resources. We also propose a sequential data transmission approach with HFCL (HFCL-SDT) to reduce the training duration. The proposed HFCL frameworks outperform previously proposed non-hybrid FL (CL) based schemes in terms of learning accuracy (communication overhead) since all the clients collaborate on the learning process with their datasets regardless of their computational resources.

Mobile and Web-of-Things (WoT) devices at the network edge account for more than half of the world's web traffic, making a great data source for various machine learning (ML) applications, particularly federated learning (FL) which offers a promising solution to privacy-preserving ML feeding on these data. FL allows edge mobile and WoT devices to train a shared global ML model under the orchestration of a central parameter server. In the real world, due to resource heterogeneity, these edge devices often train different versions of models (e.g., VGG-16 and VGG-19) or different ML models (e.g., VGG and ResNet) for the same ML task (e.g., computer vision and speech recognition). Existing FL schemes have assumed that participating edge devices share a common model architecture, and thus cannot facilitate FL across edge devices with heterogeneous ML model architectures. We explored this architecture heterogeneity challenge and found that FL can and should accommodate these edge devices to improve model accuracy and accelerate model training. This paper presents our findings and FlexiFed, a novel scheme for FL across edge devices with heterogeneous model architectures, and three model aggregation strategies for accommodating architecture heterogeneity under FlexiFed. Experiments with four widely-used ML models on four public datasets demonstrate 1) the usefulness of FlexiFed; and 2) that compared with the state-of-the-art FL scheme, FlexiFed improves model accuracy by 2.6%-9.7% and accelerates model convergence by 1.24 × -4.04 ×.

Supported by some of the major revolutionary technologies, such as Internet of Vehicles (IoVs), Edge Computing, and Machine Learning (ML), the traditional Vehicular Networks (VNs) are changing drastically and converging rapidly into one of the most complex, highly intelligent, and advanced networking systems, mostly known as Intelligent Transportation System (ITS). Recently, distributed ML techniques, such as Federated Learning (FL) have gained huge popularity mainly for their advantages in terms of intelligence sharing and privacy concerns. VNs are a natural contender for exploiting FL for solving challenging problems; however, their limited resources, dynamic nature, high speed, and reduced latency requirements often become the bottleneck. V2X communication technologies allow vehicular terminals (VTs) to share their valuable local environment parameters and become aware of their surroundings. Such information can be utilized to build a more sustainable and affordable FL platform for serving VTs. Gaining from recently introduced 3D architectures, integrating terrestrial and aerial edge computing layers, we present here a distributed FL platform able to distribute the FL process on a 3D fashion while reducing the overall communication cost for providing vehicular services. The framework is defined as a constrained optimization problem for reducing the overall FL process cost through a proper network selection between various nodes. We have modeled the FL network selection problem as a sequential decision-making process through a Markov Decision Process (MDP) with time-dependent state transition probabilities. A computation-efficient value iteration algorithm is adapted for solving the MDP. Comparison with various benchmark methods shows the overall improvement in terms of latency, energy, and FL performance.

The objective of this retrospective study is to compare three learning approaches for blood transfusion (BT) prediction after intensive care unit (ICU) admission: local learning (LL), federated learning (FL), and centralized learning (CL) across five ICU databases (eICU Collaborative Research Database, Medical Information Mart for Intensive Care IV, High-Resolution Intensive Care Unit Dataset, Amsterdam University Medical Center Database, University Hospital of Augsburg). As machine learning model we used XGBoost and included 15 clinical variables. The prediction task consists of a 3-h observation window, followed by a 2-h prediction window. We evaluated the models using internal and external validation with area under the receiver-operator curve, area under the precision-recall curve, PPV, Brier score and F1 score. CL consistently outperformed FL and LL in both internal validation (AUPRC range: 0.73-0.95 (CL) vs 0.63-0.96 (FL) and 0.69-0.96 (LL)) and external validation (AUPRC range: 0.61-0.89 (CL) vs 0.45-0.91 (FL) and 0.37-0.90 (LL)). FL showed variable performance across datasets. The complexity of the multivariable clinical prediction of BTs may create substantial challenges for FL effectiveness, particularly under high data heterogeneity conditions that are common in healthcare.

Nowadays, the integration of home automation systems with smart thermostats is a common trend, designed to enhance resident comfort and conserve energy. The introduction of smart thermostats that can run machine learning algorithms has opened the door for on-device training, enabling customized thermal experiences in homes. However, leveraging the flexibility offered by on-device learning has been hindered by the absence of a tailored learning scheme that allows for accurate on-device training of thermal models. Traditional centralized learning (CL) and federated learning (FL) schemes rely on a central server that controls the learning experience, compromising the home’s privacy and requiring significant energy to operate. To address these challenges, we propose PePTM, a personalized peer-to-peer thermal modeling algorithm that generates tailored thermal models for each home, offering a controlled learning experience with a minimal training energy footprint while preserving the home’s privacy, an aspect difficult to achieve in both CL and FL. PePTM consists of local and collaborative learning phases that enable each home to train its thermal model and collaboratively improve it with a set of similar homes in a peer-to-peer fashion. To showcase the effectiveness of PePTM, we use a year’s worth of data from US homes to train thermal models using the RNN time-series model and compare the data across three learning schemes: CL, FL, and PePTM, in terms of model performance and the training energy footprint. Our experimental results show that PePTM is significantly energy-efficient, requiring 695 and 40 times less training energy than CL and FL, respectively, while maintaining comparable performance. We believe that PePTM sets the stage for new avenues for on-device thermal model training, providing a personalized thermal experience with reduced energy consumption and enhanced privacy.

Federated learning is a framework for machine learning that is dedicated to data privacy protection. In federated learning, system cannot fully control the behavior of clients which can be faulty. These behaviors include sharing arbitrary faulty gradients and delaying the process of sharing due to Byzantine attacks or clients’ own software and hardware failures. In federated learning, the parameter server may also be faulty during gradient collection and aggregation, mainly including gradient-based training data inference and model parameter faulty update. The above problems may lead to reduced accuracy of federated learning model training, leakage of client privacy, etc. Existing research enhances the robustness of federated learning by exploiting the decentralization and immutability of Blockchain. For untrusted clients, most research is based on Byzantine fault tolerance to defend against clients indiscriminately, and may cause model accuracy reduction. In addition, most of the research focus on unencrypted gradients, and there is insufficient research on dealing with client anomalies in the case of gradient encryption. For untrusted parameter servers, existing research has problems in energy overhead and scalability. Aiming at the problems above, this paper studies the robustness of federated learning, and proposes a blockchain-based federated learning parameter update architecture PUS-FL. Through experiments simulating distributed machine learning on neural networks, we demonstrate that the anomaly detection algorithm of PUS-FL outperforms conventional gradient filters including geometric median, Multi-Krum and trimmed mean. In addition, our experiments also verify that the scalability-enhanced parameter aggregation consensus algorithm proposed in this paper(SE-PBFT) improves consensus scalability by reducing communication complexity.

With the growth of participating clients, the centralized parameter server (PS) will seriously limit the scale and efficiency of Federated Learning (FL). A straightforward approach to scale up the FL system is to construct a Parallel FL (PFL) system with multiple parallel PSes. However, it is unclear whether PFL can really accelerate FL or reduce the training time of FL. Even if the answer is yes, it is non-trivial to design a highly efficient parameter average algorithm for a PFL system. In this paper, we propose a completely parallelizable FL algorithm called P-FedAvg under the PFL architecture. P-FedAvg extends the well-known FedAvg algorithm by allowing multiple PSes to cooperate and train a learning model together. In P-FedAvg, each PS is only responsible for a fraction of total clients, but PSes can mix model parameters in a dedicatedly designed way so that the FL model can well converge. Different from heuristic-based algorithms, P-FedAvg is with theoretical guarantees. To be rigorous, we theoretically analyze the convergence rate of P-FedAvg in terms of the number of conducted iterations, the communication cost of each global iteration and the optimal weights for each PS to mix parameters with its neighbors. Based on theoretical analysis, we conduct a case study on five typical overlay topolgoies formed by PSes to further examine the communication efficiency under different topologies, and investigate how the overlay topology affects the convergence rate, communication cost and robustness of a PFL system. Lastly, we perform extensive experiments with real datasets to verify our analysis and demonstrate that P-FedAvg can significantly speed up FL than traditional FedAvg and other competitive baselines. We believe that our work can help to lay a theoretical foundation for building more efficient PFL systems.

Smart energy performance monitoring and optimisation at the supplier and consumer levels is essential to realising smart cities. In order to implement a more sustainable energy management plan, it is crucial to conduct a better energy forecast. The next-generation smart meters can also be used to measure, record, and report energy consumption data, which can be used to train machine learning (ML) models for predicting energy needs. However, sharing energy consumption information to perform centralised learning may compromise data privacy and make it vulnerable to misuse, in addition to incurring high transmission overhead on communication resources. This study addresses these issues by utilising federated learning (FL), an emerging technique that performs ML model training at the user/substation level, where data resides. We introduce FedraTrees, a new, lightweight FL framework that benefits from the outstanding features of ensemble learning. Furthermore, we developed a delta-based FL stopping algorithm to monitor FL training and stop it when it does not need to continue. The simulation results demonstrate that FedraTrees outperforms the most popular federated averaging (FedAvg) framework and the baseline Persistence model for providing accurate energy forecasting patterns while taking only 2% of the computation time and 13% of the communication rounds compared to FedAvg, saving considerable amounts of computation and communication resources.

Recent medical applications are largely dominated by the application of Machine Learning (ML) models to assist expert decisions, leading to disruptive innovations in radiology, pathology, genomics, and hence modern healthcare systems in general. Despite the profitable usage of AI-based algorithms, these data-driven methods are facing issues such as the scarcity and privacy of user data, as well as the difficulty of institutions exchanging medical information. With insufficient data, ML is prevented from reaching its full potential, which is only possible if the database consists of the full spectrum of possible anatomies, pathologies, and input data types. To solve these issues, Federated Learning (FL) appeared as a valuable approach in the medical field, allowing patient data to stay where it is generated. Since an FL setting allows many clients to collaboratively train a model while keeping training data decentralized, it can protect privacy-sensitive medical data. However, FL is still unable to deliver all its promises and meets the more stringent requirements (e.g., latency, security) of a healthcare system based on multiple Internet of Medical Things (IoMT). For example, although no data are shared among the participants by definition in FL systems, some security risks are still present and can be considered as vulnerabilities from multiple aspects. This paper sheds light upon the emerging deployment of FL, provides a broad overview of current approaches and existing challenges, and outlines several directions of future work that are relevant to solving existing problems in federated healthcare, with a particular focus on security and privacy issues.

In order to meet the extremely heterogeneous requirements of the next generation wireless communication networks, research community is increasingly dependent on using machine‐learning solutions for real‐time decision‐making and radio resource management. Traditional machine learning employs fully centralized architecture in which the entire training data is collected at one node for example, cloud server, that significantly increases the communication overheads and also raises severe privacy concerns. Toward this end, a distributed machine‐learning paradigm termed as federated learning (FL) has been proposed recently. In FL, each participating edge device trains its local model by using its own training data. Then, via the wireless channels the weights or parameters of the locally trained models are sent to the central parameter server (PS), that aggregates them and updates the global model. On one hand, FL plays an important role for optimizing the resources of wireless communication networks, on the other hand, wireless communications is crucial for FL. Thus, a “bidirectional” relationship exists between FL and wireless communications. Although FL is an emerging concept, many publications have already been published in the domain of FL and its applications for next generation wireless networks. Nevertheless, we noticed that none of the works have highlighted the bidirectional relationship between FL and wireless communications. Therefore, the purpose of this survey article is to bridge this gap in literature by providing a timely and comprehensive discussion on the interdependency between FL and wireless communications.

Federated learning (FL) is currently the most widely adopted framework for collaborative training of (deep) machine learning models under privacy constraints. Albeit its popularity, it has been observed that FL yields suboptimal results if the local clients' data distributions diverge. To address this issue, we present clustered FL (CFL), a novel federated multitask learning (FMTL) framework, which exploits geometric properties of the FL loss surface to group the client population into clusters with jointly trainable data distributions. In contrast to existing FMTL approaches, CFL does not require any modifications to the FL communication protocol to be made, is applicable to general nonconvex objectives (in particular, deep neural networks), does not require the number of clusters to be known a priori, and comes with strong mathematical guarantees on the clustering quality. CFL is flexible enough to handle client populations that vary over time and can be implemented in a privacy-preserving way. As clustering is only performed after FL has converged to a stationary point, CFL can be viewed as a postprocessing method that will always achieve greater or equal performance than conventional FL by allowing clients to arrive at more specialized models. We verify our theoretical analysis in experiments with deep convolutional and recurrent neural networks on commonly used FL data sets.

The increasing processing capabilities of massively deployed Internet of Things (IoT) and edge devices coupled with the rising demands for fast and autonomous decision-making pave the way for accurate Machine Learning (ML) model training in proximity to where data is collected, as proposed by the Federated Learning (FL) paradigm. The devices contributing to the local training (clients) are typically large in number and differ in terms of computing and battery capabilities, experienced link quality conditions and owned datasets. If clients are not properly selected by the entity in charge of aggregating the individual models (FL server) both the convergence speed and the accuracy of the resulting model can be affected. The interest in the literature for the optimization of the client selection policies is manifest, but less attention has been devoted to the design of the practical implementation of such policies with focus on the client discovery procedures. In order to fill this gap, an edge-based framework is proposed to improve the FL client discovery procedures by leveraging (i) the Message Queue Telemetry Transport (MQTT) protocol for FL clients-server interactions aimed to retrieve the capabilities of the potential clients and (ii) the Open Mobile Alliance (OMA) Lightweight-Machine-to-Machine (LwM2M) for the semantic description of such capabilities. A proof-of-concept is provided to assess the communication footprint incurred by the proposal.

The latest growth of storage capabilities has led to an accumulating volume of medical data stored locally by various healthcare entities. Given the recent progress observed in the domain of artificial intelligence, these data could be efficiently exploited, leading to improved and less expensive healthcare conditions. However, the common practice is medical data to be solely locally used, ending up poorly exploited, due to strict sharing restrictions stemming mainly from privacy limitation of their sensitive nature. Considering the centralized character of conventional machine learning approaches, it is apparent that they cannot reassure the privacy required. On the other hand, federated learning (FL) can be regarded as an upcoming and promising answer to efficient exploitation of medical data, considering its decentralized approach. In more details, FL can enable collaboration among various participating entities on the development and training of a common, central and fully shared model without need of sharing owned sensitive data. Thus, apparently FL approach not only can mitigate the privacy-preservation issues but can lead to the development of reliable and robust healthcare tools. Indicatively, FL can facilitate the development of a model capable of assessing the health risk, which can be a vital tool for medical sciences. To this end, in this work we present a tool capable of assessing the occurrence of different diseases or complications. This tool is based on FL technique utilizing deep neural network model. The FL model developed herein is indicatively applied to four different medical applications proving its generality in the healthcare domain. The FL approach discussed herein is compared with a corresponding centralized learning. According to the demonstrated results, FL can consist a useful health risk assessment tool exhibiting acceptable performance while preserving privacy in sensitive medical data.

In recent years, advances in diagnosing and classifying diseases using machine learning (ML) have grown exponentially. However, due to the many privacy regulations regarding personal data, pooling together data from multiple sources and storing them in a single (centralized) location for traditional ML model training are often infeasible. Federated learning (FL), a collaborative learning paradigm, can sidestep this major pitfall by creating a global ML model that is trained by aggregating model weights from individual models that are separately trained on their own data silos, therefore avoiding most data privacy concerns. This study addresses the centralized data issue with FL by applying a novel DataWeightedFed architectural approach for effective fundus disease diagnosis from ophthalmic images. It includes a novel method for aggregating model weights by comparing the size of each model’s data and taking a dynamically weighted average of all the model’s weights. Experimental results showed a small average 1.85% loss in accuracy when training using FL compared to centralized ML model systems, a nearly 92% improvement over the conventional 55% accuracy loss. The obtained results demonstrate that this study’s FL architecture can maximize both privacy preservation and accuracy for ML in fundus disease diagnosis and provide a secure, collaborative ML model training solution within the eye healthcare space.

Federated learning (FL) has been vigorously promoted in wireless edge networks as it fosters collaborative training of machine learning (ML) models while preserving individual user privacy and data security. In conventional FL, user equipment (UE) and an aggregator can collaboratively train shared global ML models by transmitting interactive ML models. In wireless edge networks, heterogeneity of multi-dimensional resources (e.g., computing and communication resources) used to train and transmit FL models may introduce stragglers, characterized by a slow update and/or transmission of local models. The stragglers can significantly degrade learning performance of FL, as the slowest participating UE can dramatically slow down the entire convergence. In this paper, to alleviate the negative impact of stragglers, we propose a dynamic straggler-aware clustering based FL (FeDSC) mechanism via adaptive UE clustering. Specifically, we first group participating UEs into multiple clusters based on their available computing and wireless resources. Then, we propose an adaptive clustering scheme to synchronously update the cluster aggregation model. Meanwhile, an edge server performs global aggregation of different cluster models in an asynchronous manner. Finally, we theoretically demonstrate the convergence of our proposed mechanism via numerical results. Numerical results show that our proposed mechanism can effectively reduce training time and wireless bandwidth consumption, while improving training efficiency and guaranteeing learning accuracy.