ESFL: Efficient Split Federated Learning Over Resource-Constrained Heterogeneous Wireless Devices

In this paper, the optimization of network performance to support the deployment of federated learning (FL) is investigated. In particular, in the considered model, each user owns a machine learning (ML) model by training through its own dataset, and then transmits its ML parameters to a base station (BS) which aggregates the ML parameters to obtain a global ML model and transmits it to each user. Due to limited radio frequency (RF) resources, the number of users that participate in FL is restricted. Meanwhile, each user uploading and downloading the FL parameters may increase communication costs thus reducing the number of participating users. To this end, we propose to introduce visible light communication (VLC) as a supplement to RF and use compression methods to reduce the resources needed to transmit FL parameters over wireless links so as to further improve the communication efficiency and simultaneously optimize wireless network through user selection and resource allocation. This user selection and bandwidth allocation problem is formulated as an optimization problem whose goal is to minimize the training loss of FL. We first use a model compression method to reduce the size of FL model parameters that are transmitted over wireless links. Then, the optimization problem is separated into two subproblems. The first subproblem is a user selection problem with a given bandwidth allocation, which is solved by a traversal algorithm. The second subproblem is a bandwidth allocation problem with a given user selection, which is solved by a numerical method. The ultimate user selection and bandwidth allocation are obtained by iteratively compressing the model and solving these two subproblems. Simulation results show that the proposed FL algorithm can improve the accuracy of object recognition by up to 16.7% and improve the number of selected users by up to 68.7%, compared to a conventional FL algorithm using only RF.

In this paper, the convergence time of federated learning (FL), when deployed over a realistic wireless network, is studied. In particular, a wireless network is considered in which wireless users transmit their local FL models (trained using their locally collected data) to a base station (BS). The BS, acting as a central controller, generates a global FL model using the received local FL models and broadcasts it back to all users. Due to the limited number of resource blocks (RBs) in a wireless network, only a subset of users can be selected to transmit their local FL model parameters to the BS at each learning step. Moreover, since each user has unique training data samples, the BS prefers to include all local user FL models to generate a converged global FL model. Hence, the FL training loss and convergence time will be significantly affected by the user selection scheme. Therefore, it is necessary to design an appropriate user selection scheme that can select the users who can contribute toward improving the FL convergence speed more frequently. This joint learning, wireless resource allocation, and user selection problem is formulated as an optimization problem whose goal is to minimize the FL convergence time and the FL training loss. To solve this problem, a probabilistic user selection scheme is proposed such that the BS is connected to the users whose local FL models have significant effects on the global FL model with high probabilities. Given the user selection policy, the uplink RB allocation can be determined. To further reduce the FL convergence time, artificial neural networks (ANNs) are used to estimate the local FL models of the users that are not allocated any RBs for local FL model transmission at each given learning step, which enables the BS to improve the global model, the FL convergence speed, and the training loss. Simulation results show that the proposed approach can reduce the FL convergence time by up to 56% and improve the accuracy of identifying handwritten digits by up to 3%, compared to a standard FL algorithm.

Federated learning (FL) has various advantages, including reduced communication overhead and improved privacy protection for situations involving frequent data production. FL involves training local models on end devices before transferring them to the cloud or network edge for global aggregation. End-devices can use this aggregated model to enhance their local models. Until convergence is reached, this iterative procedure is carried out repeatedly. FL has many benefits, but it also has several drawbacks. Constraints in computing resources are the most notable. In general, end-devices lack the computational capacity to learn local models effectively. To solve this issue, the split FL (SFL) was developed. However, wireless resource limitations and uncertainty also make SFL difficult to enable. For SFL at the network edge, we propose a joint task-offloading, resource allocation, and end-devices computing resources optimization problem. Moreover, we consider the quality of service (QoS) constraint in terms of latency for SFL. Because our problem involves both continuous and binary variables, it is a mixed-integer non-linear programming problem and is challenging to solve. We provide a solution based on optimization and a double deep Q-network (DDQN) with dueling. We use comprehensive simulations to validate the proposed approach. Our proposed scheme, namely, DDQN+dueling, outperforms traditional DDQN-based schemes in terms of faster convergence and attaining QoS for various configurations.

Federated learning (FL) enables privacy-preserving distributed machine learning by training models on edge client devices using their local data without revealing their raw data. In edge environments, various applications require different neural network models, making it crucial to perform joint training of multiple models on edge devices, known as multi-task FL. While existing multi-task FL approaches enhance resource utilization on edge devices through adaptive resource configuration or client selection, optimizing either of these aspects alone may lead to suboptimality. Therefore, in this paper, we explore a joint client selection and resource configuration method called JCSRC for multi-task FL, aiming to maximize energy efficiency in environments with limited computation and communication resources and heterogeneous client devices. Firstly, we formalize this problem as a mixed-integer nonlinear programming problem considering all these characteristics and prove its NP-hardness. To address this problem, we first design a multi-agent reinforcement learning (MARL)-based client selection method that selects appropriate clients for each task to train their models. The MARL method makes client selection decisions based on the clients’ data quality, energy efficiency, communication, and computation capacity to ensure fast convergence and energy efficiency. Then, we design a particle swarm optimization (PSO)-based resource configuration scheme that configures appropriate computation and bandwidth resources for each task on each client. The PSO scheme makes resource configuration decisions based on theoretically derived optimal CPU frequency and bandwidth to achieve high energy efficiency. Finally, we carry out extensive simulations and testbed-based experiments to validate our proposed JCSRC. The results demonstrate that, in comparison to state-of-the-art solutions, JCSRC can save energy consumption by up to 59% to achieve the target accuracy.

A novel federated learning-based two-stage approach for ship energy consumption optimization considering both shipping data security and statistical heterogeneity

Federated learning (FL) has gained attention as a distributed learning paradigm for its data privacy benefits and accelerated convergence through parallel computation. Traditional FL relies on a server-client (SC) architecture, where a central server coordinates multiple clients to train a global model, but this approach faces scalability challenges due to server communication bottlenecks. To overcome this, the ring-all-reduce (RAR) architecture has been introduced, eliminating the central server and achieving bandwidth optimality. However, the tightly coupled nature of RAR's ring topology exposes it to unique Byzantine attack risks not present in SC-based FL. Despite its potential, designing Byzantine-robust RAR-based FL algorithms remains an open problem. To address this gap, we propose BRACE (<u>B</u>yzantine-<u>r</u>obust ring-<u>a</u>ll-redu<u>ce</u>), the first RAR-based FL algorithm to achieve both Byzantine robustness and communication efficiency. We provide theoretical guarantees for the convergence of BRACE under Byzantine attacks, demonstrate its bandwidth efficiency, and validate its practical effectiveness through experiments. Our work offers a foundational understanding of Byzantine-robust RAR-based FL design.

Motivated by the increasingly powerful computing capabilities of end-user equipment, and by the growing privacy concerns over sharing sensitive raw data, a distributed machine learning paradigm known as federated learning (FL) has emerged. By training models locally at each client and aggregating learning models at a central server, FL has the capability to avoid sharing data directly, thereby reducing privacy leakage. However, the conventional FL framework relies heavily on a single central server, and it may fail if such a server behaves maliciously. To address this single point of failure, in this work, a blockchain-assisted decentralized FL framework is investigated, which can prevent malicious clients from poisoning the learning process, and thus provides a self-motivated and reliable learning environment for clients. In this framework, the model aggregation process is fully decentralized and the tasks of training for FL and mining for blockchain are integrated into each participant. Privacy and resource-allocation issues are further investigated in the proposed framework, and a critical and unique issue inherent in the proposed framework is disclosed. In particular, a lazy client can simply duplicate models shared by other clients to reap benefits without contributing its resources to FL. To address these issues, analytical and experimental results are provided to shed light on possible solutions, i.e., adding noise to achieve local differential privacy and using pseudo-noise (PN) sequences as watermarks to detect lazy clients.

Machine learning (ML) and Deep learning (DL) models are popular in many areas, from business, medicine, industries, healthcare, transportation, smart cities, and many more. However, the conventional centralized training techniques may not apply to upcoming distributed applications, which require high accuracy and quick response time. It is mainly due to limited storage and performance bottleneck problems on the centralized servers during the execution of various ML and DL-based models. However, federated learning (FL) is a developing approach to training ML models in a collaborative and distributed manner. It allows the full potential exploitation of these models with unlimited data and distributed computing power. In FL, edge computing devices collaborate to train a global model on their private data and computational power without sharing their private data on the network, thereby offering privacy preservation by default. But the distributed nature of FL faces various challenges related to data heterogeneity, client mobility, scalability, and seamless data aggregation. Moreover, the communication channels, clients, and central servers are also vulnerable to attacks which may give various security threats. Thus, a structured vulnerability and risk assessment are needed to deploy FL successfully in real-life scenarios. Furthermore, the scope of FL is expanding in terms of its application areas, with each area facing different threats. In this paper, we analyze various vulnerabilities present in the FL environment and design a literature survey of possible threats from the perspective of different application areas. Also, we review the most recent defensive algorithms and strategies used to guard against security and privacy threats in those areas. For a systematic coverage of the topic, we considered various applications under four main categories: space, air, ground, and underwater communications. We also compared the proposed methodologies regarding the underlying approach, base model, datasets, evaluation matrices, and achievements. Lastly, various approaches’ future directions and existing drawbacks are discussed in detail.

HierFedPDP:Hierarchical federated learning with personalized differential privacy

Sixth-Generation (6G)-based Internet of Everything applications (e.g. autonomous driving cars) have witnessed a remarkable interest. Autonomous driving cars using federated learning (FL) has the ability to enable different smart services. Although FL implements distributed machine learning model training without the requirement to move the data of devices to a centralized server, it its own implementation challenges such as robustness, centralized server security, communication resources constraints, and privacy leakage due to the capability of a malicious aggregation server to infer sensitive information of end-devices. To address the aforementioned limitations, a dispersed federated learning (DFL) framework for autonomous driving cars is proposed to offer robust, communication resource-efficient, and privacy-aware learning. A mixed-integer non-linear programming (MINLP) optimization problem is formulated to jointly minimize the loss in federated learning model accuracy due to packet errors and transmission latency. Due to the NP-hard and non-convex nature of the formulated MINLP problem, we propose the Block Successive Upper-bound Minimization (BSUM) based solution. Furthermore, the performance comparison of the proposed scheme with three baseline schemes has been carried out. Extensive numerical results are provided to show the validity of the proposed BSUM-based scheme.

Within the federated learning (FL) framework, the client collaboratively trains the model in coordination with a central server, while the training data can be kept locally on the client. Thus, the FL framework mitigates the privacy disclosure and costs related to conventional centralized machine learning. Nevertheless, current surveys indicate that FL still has problems in terms of communication efficiency and privacy risks. In this paper, to solve these problems, we develop an FL framework with communication-efficient and privacy-preserving (FLCP). To realize the FLCP, we design a novel compression algorithm with efficient communication, namely, adaptive weight compression FedAvg (AWC-FedAvg). On the basis of the non-independent and identically distributed (non-IID) and unbalanced data distribution in FL, a specific compression rate is provided for each client, and homomorphic encryption (HE) and differential privacy (DP) are integrated to provide demonstrable privacy protection and maintain the desirability of the model. Therefore, our proposed FLCP smoothly balances communication efficiency and privacy risks, and we prove its security against “honest-but-curious” servers and extreme collusion under the defined threat model. We evaluate the scheme by comparing it with state-of-the-art results on the MNIST and CIFAR-10 datasets. The results show that the FLCP performs better in terms of training efficiency and model accuracy than the baseline method.

Federated Learning (FL) has emerged as a suitable option for collaborative, user-centric machine learning while protecting sensitive user data. FL differs significantly from traditional centralized frameworks that collect raw data on a central server. FL focuses on collecting model updates generated locally on edge devices. With traditional FL approaches, data must be downloaded to a central server for model training. In contrast, training occurs on the edge and only model updates are sent. This decentralized framework, unlike traditional FL approaches, is ideal for ubiquitous applications—systems woven seamlessly into everyday surroundings like smart homes, advanced mobile devices, and IoT systems—where sensitive data is constantly and automatically generated. In these systems, user privacy is paramount, and advanced methods must be implemented to ensure privacy and system efficacy. This paper discusses the integration of FL into ubiquitous environments, addressing the relevance, advantages, and disadvantages of this approach. I’d like to focus on privacy-preserving methods, but their preservation against data leakage and adversary attacks adds significant value. There are case studies to show the challenges encountered, lessons learned, and the use of FL within privacy-centric ubiquitous systems. In closing this paper, I summarize the key points and underscore the transformational potential of FL for privacy-sensitive, innovative systems. I also promote further research in this area to support its practical relevance. This paper demonstrates that among the many approaches to privacy within pervasive computing, FL is the only one that is practical and scalable.

Motivated by the rapidly increasing computing performance of devices and the abundance of device-generated data, federated learning (FL) has emerged as a new distributed machine learning (ML) scheme with a wide range of applications. However, it is well-known that FL might be severely degraded by communication overhead, as it heavily relies on communication between clients and a central server. To overcome this communication bottleneck, the wireless communication community has explored AirComp FL, applying Over-the-Air Computation (AirComp) for model aggregation. In this paper, we introduce a novel AirComp FL algorithm, A-FedCS, which utilizes count-sketch (CS) for model compression. A-FedCS exhibits scalability, addressing challenges faced by existing approaches struggling with scarce channel resources or rarely revisiting clients. Experimental results demonstrate that the proposed scheme outperforms state-of-the-art schemes, including CA-DSGD and D-DSGD. We show that the improvement is more significant in stateless FL through experiments with various settings of tasks, transmission power, bandwidth, and the number of clients. Additionally, we provide a mathematical analysis of A-FedCS by deriving its convergence rate.

Federated learning (FL) has emerged for solving the problem of data fragmentation and isolation in machine learning based on privacy protection. Each client node uploads the trained model parameter information to the central server based on the local training data, and the central server aggregates the parameter information to achieve the purpose of common training. In the real environment, the distribution of data among nodes is often inconsistent. By analyzing the influence of independent identically distributed data (non‐IID) on the accuracy of FL, it is shown that the accuracy of the model obtained by the traditional FL method is low. Therefore, we proposed the diversified sampling strategies to simulate the non‐IID data situation and came up with the OPTICS (ordering points to identify the clustering structure)‐based clustering optimization federated learning method (OCFL), which solves the problem that the learning accuracy is reduced when the data of different nodes are non‐IID in FL. Experiments indicate that OCFL greatly improves the model accuracy and training speed compared with the traditional FL algorithm.

This paper proposes a novel scheme for cell-free massive multiple-input multiple-output (CFmMIMO) networks to support any federated learning (FL) framework. This scheme allows each instead of all the iterations of the FL framework to happen in a large-scale coherence time to guarantee a stable operation of an FL process. To show how to optimize the FL performance using this proposed scheme, we consider an existing FL framework as an example and target FL training time minimization for this framework. An optimization problem is then formulated to jointly optimize the local accuracy, transmit power, data rate, and users' processing frequency. This mixed-timescale stochastic nonconvex problem captures the complex interactions among the training time, and transmission and computation of training updates of one FL process. By employing the online successive convex approximation approach, we develop a new algorithm to solve the formulated problem with proven convergence to the neighbourhood of its stationary points. Our numerical results confirm that the presented joint design reduces the training time by up to 55% over baseline approaches. They also show that CFmMIMO here requires the lowest training time for FL processes compared with cell-free time-division multiple access massive MIMO and collocated massive MIMO.