Adaptive Gradient Compression and Differential Privacy for Resource-Constrained Edge Federated Learning

This artilce aims to develop a differential private federated learning (FL) scheme with the least artificial noises added while minimizing the energy consumption of participating mobile devices. By observing that some communication efficient FL approaches and even the nature of wireless communications contribute to the differential privacy (DP) preservation of training data on mobile devices, in this paper, we propose to jointly leverage gradient compression techniques (i.e., gradient quantization and sparsification) and additive white Gaussian noises (AWGN) in wireless channels to develop a piggyback DP approach for FL over mobile devices. Even with the piggyback DP approach, information distortion caused by gradient compression and noise perturbation may slow down FL convergence, which in turn consumes more energy of mobile devices for local computing and model update communications. Thus, we theoretically analyze FL convergence and formulate an energy efficient FL optimization under piggyback DP, transmission power, and FL convergence constraints. Furthermore, we propose an efficient iterative algorithm where closed-form solutions for artificial DP noise and power control are derived. Extensive simulation and experimental results demonstrate the effectiveness of the proposed scheme in terms of energy efficiency and privacy preservation.

With the continuous improvement of hardware computing power, edge computing of industrial data has been gradually applied. In the past decade, the promotion of edge computing has also greatly improved the efficiency of industrial production. Compared with the conventional cloud computing, it not only saves the bandwidth consumption of data transmission, but also ensures the terminal data security to a certain extent. However, the continuous update of attack types also put forward new requirements for the privacy protection of industrial edge computing. So it should fundamentally solve the risk of industrial data leakage in the process of deep model training in edge terminal. In this article, we propose a new federated edge learning framework based on hybrid differential privacy and adaptive compression for industrial data processing. Specifically, it first completes the adaptive gradient compression preparation, then constructs the industrial federated learning model, and finally makes use of adaptive differential privacy model to optimize, so as to complete the privacy protection towards the transmission of gradient parameters in industrial environment. By optimizing the hybrid differential privacy and adaptive compression, we can better prevent the terminal data privacy against inference attacks. The experimental results show that this method is very effective in the industrial edge computing situation, and it also opens up a new direction for the effect of differential privacy in federated learning.

Normalization is an important but understudied challenge in privacy-related application domains such as federated learning (FL), differential privacy (DP), and differentially private federated learning (DP-FL). While the unsuitability of batch normalization for these domains has already been shown, the impact of other normalization methods on the performance of federated or differentially private models is not well-known. To address this, we draw a performance comparison among layer normalization (LayerNorm), group normalization (GroupNorm), and the recently proposed kernel normalization (KernelNorm) in FL, DP, and DP-FL settings. Our results indicate LayerNorm and GroupNorm provide no performance gain compared to the baseline (i.e. no normalization) for shallow models in FL and DP. They, on the other hand, considerably enhance the performance of shallow models in DP-FL and deeper models in FL and DP. KernelNorm, moreover, significantly outperforms its competitors in terms of accuracy and convergence rate (or communication efficiency) for both shallow and deeper models in all considered learning environments. Given these key observations, we propose a kernel normalized ResNet architecture called KNResNet-13 for differentially private learning. Using the proposed architecture, we provide new state-of-the-art accuracy values on the CIFAR-10 and Imagenette datasets, when trained from scratch.

Federated learning is a popular distributed machine learning paradigm that enables collaborative model training at multiple entities via exchanging intermediate learning results. Security and communication efficiency are crucial for successful applications of federated learning in various privacy-sensitive services. However, existing work focused on gradient defense and communication efficiency separately, and also incurred additional computation, signaling, and accuracy overhead. A lightweight (in terms of time-complexity and signaling) technique that simultaneously achieves security and communication efficiency is critical for massive resource-constrained devices (e.g., Internet-of-Things generating the data), but has yet to be established. This paper proposes a secure and efficient federated learning framework with provable communication-accuracy-security performance guarantees. A low-complexity and signaling-free stochastic quantization module is added at the client side that quantizes the original local gradients to discrete values for communication-efficient global aggregation. The stochastic quantization module is shown to be interpreted as triangular or Gaussian-multiply-triangular noises under uniform or Gaussian distributions of local gradients, hence protecting data privacy. We prove that the proposed framework exhibits an { <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (log <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> 1/δ), <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (δ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ), <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (1/δ)}-tradeoff between the communication overhead, model accuracy, and data protection, where δ is an adjustable quantization interval. Experimental results validate the tradeoff and the superiority of the proposed stochastic quantization technique in terms of communication efficiency (only 14.1% of differential privacy and 0.2% of homomorphic encryption) and computation complexity (similar to differential privacy and only 0.03% of homomorphic encryption). Under the same data protection performance, the proposed approach also outperforms (in terms of accuracy) differential privacy in all the 9 comparison settings on CIFAR10 dataset.

Privacy-preserving Federated Learning (FL) based on Differential Privacy (DP) protects clients’ data by adding DP noise to samples’ gradients and has emerged as a de facto standard for data privacy in FL. However, the accuracy of global models in DP-based FL may be reduced significantly when rogue clients occur who deviate from the preset DP-based FL approaches and selfishly inject excessive DP noise beyond expectations, thereby applying a smaller privacy budget in the DP mechanism to ensure a higher level of security. Existing DP-based FL fails to prevent such attacks as they are imperceptible. Under the DP-based FL system and random Gaussian noise, the local model parameters of the rogue clients and the honest clients have identical distributions. In particular, the rogue local models show a low performance, but directly filtering out lower-performance local models compromises the generalizability of global models, as local models trained on scarce data also behave with low performance in the early epoch. In this paper, we propose ReFL, a novel privacy-preserving FL system that enforces DP and avoids the accuracy reduction of global models caused by excessive DP noise of rogue clients. Based on the observation that rogue local models with excessive DP noise and honest local models trained on scarce data have different performance patterns in long-term training epochs, we propose a long-term contribution incentives scheme to evaluate clients’ reputations and identify rogue clients. Furthermore, we design a reputation-based aggregation to avoid the damage of rogue clients’ models on the global model accuracy, based on the incentive reputation. Extensive experiments demonstrate ReFL guarantees the global model accuracy performance 0.77% - 81.71% higher than existing DP-based FL methods in the presence of rogue clients.

The current AI revolution originated from the impressive deep learning methodology. Deep neural networks and in particular recent foundation models rely heavily on vast amounts of data and computing power, which poses the question of how to efficiently train large AI models from big data distributed geographically in multiple organisations. Because of privacy concerns and commercial interest of the data providers, as well as mandatory regulation rules such as General Data Protection Regulation (GDPR) of the European Union, sensitive local data must not be leaked to others. This restriction makes traditional distributed learning paradigms an unfeasible solution in which local information can freely flow among nodes. Federated learning is an emerging distributed learning paradigm that features privacy protection by utilising sophisticated algorithms such as differential privacy and aggregation schemes to generate global results. However, while federated learning has made great progress and been deployed in many industrial systems, there remain obstacles in the canonical paradigm that prevent its advance in the era of large models. In this thesis, two important difficulties are identified: low training efficiency and lack of explainability. Through experimental results from the literature and collected operating stat- istics of deployed federated systems, the main training bottleneck boils down to the transfer time of large messages, which could add up to more than one day in a typical federated training across distant sites. On the other hand, explainability can help evaluate the quality of training samples, shorten training time, and facilitate the acceptance of complex AI models. These advantages are valuable in federated environments because there is no authoritative control over the data quantity of participants. Motivated by the aforementioned challenges of large-scale federated learning, the topic of this thesis is efficient and explainable large-scale federated learning. An extensive investigation of the literature and our works on the topic are presented. Specifically, our contributions are as follows: • We propose Hy pergradient for Data Relevance Analysis (HyDRA), which is an explainable approach that interprets the predictions made by deep neural networks as effects of training samples. Existing approaches like Influence Function generally estimate training sample contributions around the final model parameters and ignore how the training samples shape the optimisation trajectory. In contrast, HyDRA assesses the contribution of training samples to test samples throughout the optimisation trajectory. In order to accelerate computation, we provide an approximation formula and prove that, under moderate conditions, the approximation error is bounded. Consistent with the theoretical claim, empirical results indicate that the error is indeed small. In addition, we quantitatively demonstrate that HyDRA outperforms Influence Function in accurately estimating data contribution and detecting noisy data labels. • We propose the Guided T runcation G radient Shapley (GTG-Shapley) ap- proach to fairly evaluate the contributions of the participants to the perform- ance of the final model without exposing their private data. To sustain the long-term operation of a federated learning (FL) ecosystem, it is important to attract high-quality data owners with appropriate incentive schemes. Fair evaluation of participant contributions is an important building block of such incentive schemes. Although Shapley value-based techniques have been widely adopted to provide fair evaluation in such cases, existing approaches incur a significant computational cost, making them difficult to apply in practice. GTG-Shapley approximates sub-coalitions by aggregating sub- models, rather than repeatedly training from scratch. It also uses Monte Carlo sampling and other optimisations to further reduce the number of sub- coalition evaluations. The experimental results showed that GTG-Shapley can closely approximate the actual Shapley value, while significantly increasing the computational efficiency compared to other advanced methods, especially in non-i.i.d. settings. • We propose Fed erated O pportunistic B lock D ropout (FedOBD), which provides an efficient framework for federated learning. The key novelty is that it decomposes large neural networks into semantic blocks so that federated participants can opportunistically upload quantised blocks, which can greatly speed up training. Extensive experiments evaluating FedOBD against four competitive approaches based on common datasets show that it reduces the overall communication overhead by more than 88% compared to the best-performing baseline approach, while achieving the highest test accuracy. FedOBD was deployed in industry and helped the company reduce the training communication overhead by more than 70% compared to its previous AI engine, while maintaining model performance at a test F1 score of over 85%. Finally, we discuss further research directions, including: (1) integrating the pro- posed methods into a unified pipeline that provides hierarchical explanations at both sample and participant levels while maintaining communication efficiency; (2) extending the efficient communication techniques to federated graph learning, where convolutional operations over distributed sub-graphs incur high communication costs; and (3) balancing explainability and productivity in deployed industrial AI systems.

In domains such as remote healthcare, where sensitive data must be protected, Federated Learning (FL) has emerged as a radical new way to build privacy-preserving AI models. In contrast, centralized AI systems have always existed, aggregating patient data into a single repository, subject to vulnerabilities in privacy, security, and regulatory compliance. In this study, we propose a novel FL implementation in remote healthcare that supports distributed training across multiple devices and remote healthcare facilities, while preserving patient privacy. We design a hypothetical framework taking advantage of the state of the art in machine learning tools including TensorFlow Federated and privacy enhancing technologies like differential privacy and secure aggregation. We evaluate the effectiveness of FL for simulating patient vitals, symptoms, and outcomes for different healthcare institutions in the research. This shows that FL can get comparable model accuracy with centralized systems, and its privacy and scalability aspect gains much more. Communication overhead and data heterogeneity are discussed, and practical strategies around their mitigation are laid out for the practical deployment of this method. In this work, we offer a comprehensive analysis of the application of FL in the healthcare domain, particularly on how it can contribute to the security of FL-sensitive data and the development of medical AI applications. Future research direction in building privacy-preserving AI models specific to the fluctuating needs of remote healthcare environments is now feasible with these results.

In recent years, to surmount the challenges of data sensitivity and data silos, federated learning (FL), a collaborative decentralized privacy preservation machine learning framework was proposed and rapidly became a current research hotspot. In this paper, we present an overview of FL workflow, as well as the communication overhead and privacy leakage problems associated with FL. In order to alleviate these two pain points, several corresponding solutions are presented in this paper along with their advantages and disadvantages. To reduce the communication overhead, this paper suggests three reliable approaches by reviewing related research including increasing the amount of computing on the edge side, model compression, and improving parallelism. To address the privacy concerns, this paper introduces three prevailing privacy preservation techniques: differential privacy, homomorphic encryption, and secure multiparty computation (SMC), and shows how to efficiently incorporate these privacy-preserving techniques into a federated learning framework by summarizing relevant literature. On the whole, this paper contributes to explaining the overall workflow of a FL framework and helps advance the understanding of FL in improving communication efficiency and enhancing privacy security.

The convergence of federated learning (FL) and differential privacy (DP) presents a transformative approach to secure predictive modeling in healthcare finance, where safeguarding sensitive patient and financial data is paramount. Traditional centralized machine learning methods often raise significant privacy concerns due to the necessity of aggregating data from multiple institutions. Federated learning mitigates this by enabling decentralized model training across disparate data sources, such as hospitals, insurance firms, and financial institutions, without exposing raw data. However, FL alone remains vulnerable to inference and reconstruction attacks. To enhance security, differential privacy introduces mathematically rigorous noise mechanisms that obfuscate sensitive information while preserving data utility. This paper explores the synergistic integration of DP and FL for building robust, privacy-preserving predictive models tailored to healthcare finance applications, such as fraud detection, insurance risk scoring, billing optimization, and cost forecasting. We discuss the architectural design, privacy-utility trade-offs, and implementation challenges involved, including issues of scalability, model accuracy, regulatory compliance (e.g., HIPAA and GDPR), and communication overhead. Furthermore, real-world use cases and simulation results demonstrate the efficacy of DP-FL frameworks in delivering secure and accurate predictive insights without compromising individual or institutional privacy. The study concludes by highlighting open research directions and recommending best practices for deploying privacy-enhanced federated learning systems in complex, multi-stakeholder healthcare financial ecosystems.

The convergence of vehicular technology, artificial intelligence (AI), and distributed computing has catalyzed the emergence of the Internet of Vehicles (IoV) as a cornerstone of next-generation intelligent transportation systems (ITS). By enabling vehicle-to-everything (V2X) communication, IoV supports cooperative perception, real-time decision-making, and autonomous driving. However, the reliance on large-scale, data-driven intelligence in IoV exposes systems to critical challenges, including adversarial poisoning, privacy leakage, identity forgery, and the fragility of centralized learning architectures. Federated Learning (FL) has been proposed as a promising paradigm to alleviate some of these issues by enabling distributed model training without centralizing sensitive vehicular data. Nonetheless, conventional FL remains vulnerable to security and trust limitations, particularly in dynamic vehicular environments. This thesis addresses these challenges by designing secure, privacy-preserving, and scalable FL frameworks that leverage distributed ledger technologies and cutting-edge security mechanisms.The thesis advances knowledge through four interconnected contributions. First, two novel optimization-driven poisoning attack models are introduced: PA-PSOSA and PAPSOGA, which combine particle swarm optimization with simulated annealing and genetic algorithms, respectively. These models demonstrate that even a small poisoning budget can substantially degrade global model utility under black-box and clean-label constraints, highlighting the urgency of robust defenses in vehicular FL. Second, a permissioned blockchain-enabled FL (BCFL) framework is proposed, in which consortium edge nodes running Practical Byzantine Fault Tolerance (PBFT) consensus replace the central aggregator. With blockchain integration and data validation mechanisms, this design ensures identity authentication, verifiable audit trails, and improved resilience against poisoning and Sybil attacks, while maintaining high model accuracy under adversarial conditions. Third, the framework is further enhanced to achieve inference-resistance by integrating secure aggregation (SecAgg) and differential privacy (DP), and lightweight with off-chain commitments. This design significantly reduces ledger storage requirements, increases system throughput, and mitigates inference-based privacy risks. Finally, to overcome the scalability limitations of PBFT-based BCFL, a DAG-enabled FL (DFL) framework is developed. By leveraging parallel validation, utility-score-based tip selection, and reputation-weighted aggregation, this framework significantly improves scalability, reduces communication complexity, and enhances robustness in asynchronous vehicular environments.Together, these contributions articulate a coherent progression from exposing vulnerabilities in vehicular FL to constructing secure, privacy-preserving, and scalable frameworks tailored for IoV ecosystems. The findings demonstrate that interdisciplinary integration of optimization theory, cryptography, differential privacy, and distributed ledger technologies is indispensable for trustworthy vehicular intelligence. Beyond theoretical significance, the proposed frameworks offer practical designs for deployment in safety-critical IoV environments. Future research directions include the integration of zero-knowledge proofs (ZKP) for verifiable privacy, adaptive defenses against evolving adversarial strategies, and experimental validation in real-world vehicular testbeds. Collectively, this thesis establishes a foundation for secure federated intelligence in IoV, contributing to the reliability, efficiency, and trustworthiness of next-generation ITS.

Within the federated learning (FL) framework, the client collaboratively trains the model in coordination with a central server, while the training data can be kept locally on the client. Thus, the FL framework mitigates the privacy disclosure and costs related to conventional centralized machine learning. Nevertheless, current surveys indicate that FL still has problems in terms of communication efficiency and privacy risks. In this paper, to solve these problems, we develop an FL framework with communication-efficient and privacy-preserving (FLCP). To realize the FLCP, we design a novel compression algorithm with efficient communication, namely, adaptive weight compression FedAvg (AWC-FedAvg). On the basis of the non-independent and identically distributed (non-IID) and unbalanced data distribution in FL, a specific compression rate is provided for each client, and homomorphic encryption (HE) and differential privacy (DP) are integrated to provide demonstrable privacy protection and maintain the desirability of the model. Therefore, our proposed FLCP smoothly balances communication efficiency and privacy risks, and we prove its security against “honest-but-curious” servers and extreme collusion under the defined threat model. We evaluate the scheme by comparing it with state-of-the-art results on the MNIST and CIFAR-10 datasets. The results show that the FLCP performs better in terms of training efficiency and model accuracy than the baseline method.

Federated learning (FL), as a distributed learning mechanism, can have model training completed without directly uploading original data, effectively reducing the risk of privacy leakage. However, through the shared gradient information, research shows that adversaries may reconstruct the original data. To further protect the privacy of federated learning, a federated learning defense scheme is proposed based on generative adversarial networks (GAN), which is combined with adaptive differential privacy. Firstly, the real data distribution features are learned through GAN, and replaceable pseudo data are generated. Then, the pseudo data are added with adaptive noise. Finally, the pseudo gradient generated by the pseudo data in the model is used to replace the real gradient so that adversaries cannot obtain the real gradient to further protect the privacy of user data. After simulation experiments are carried out on the MNIST dataset, the algorithm is verified using the gradient attack method. The experimental results show that the proposed algorithm is superior to the federated learning algorithm based on differential privacy in accuracy. Compared with the FedAvg algorithm, only 0.48% accuracy is lost. Therefore, it achieves a good balance between algorithm accuracy and data privacy.

As a novel distributed machine learning approach, federated learning (FL) is proposed to train a global model while preserving data privacy. However, some studies manifest that adversaries can still recover private information from the gradients. Differential privacy (DP) is a rigorous mathematical tool to protect records in a database against leakage. It has been widely applied in FL by perturbing the gradients. Nevertheless, while using DP in FL, the convergence performance of the global model is inevitably degraded. In this paper, we implement a DP-based FL scheme, which achieves local DP (LDP) by adding well-designed Gaussian noise on the gradients before clients upload them to the server. After that, we propose two strategies to improve the convergence performance of the DP-based FL. Both methods are realized by modifying the local objective function to limit the effect of LDP noise on convergence without degrading the privacy protection level. We then provide the detailed framework which adopts the LDP scheme and two strategies. The framework on different machine learning models is tested by simulation results, which show that our framework can improve the convergence performance up to 40% faster under different noise compared with other DP-based FL. Finally, we show the theoretical convergence guarantee of our proposed framework by first presenting the expected decrease in the global loss function for one round of training and then providing the upper convergence bound after multiple communication rounds.

Federated Learning (FL) has recently received a lot of attention for large-scale privacy-preserving machine learning. However, high communication overheads due to frequent gradient transmissions decelerate FL. To mitigate the communication overheads, two main techniques have been studied: (i) local update of weights characterizing the trade-off between communication and computation and (ii) gradient compression characterizing the trade-off between communication and precision. To the best of our knowledge, studying and balancing those two trade-offs <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">jointly and dynamically</i> while considering their impacts on convergence has remained unresolved even though it promises significantly faster FL. In this paper, we first formulate our problem to minimize learning error with respect to two variables: <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">local update coefficients</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">sparsity budgets</i> of gradient compression who characterize trade-offs between communication and computation/precision, respectively. We then derive an upper bound of the learning error in a given wall-clock time considering the interdependency between the two variables. Based on this theoretical analysis, we propose an enhanced FL scheme, namely Fast FL (FFL), that jointly and dynamically adjusts the two variables to minimize the learning error. We demonstrate that FFL consistently achieves higher accuracies faster than similar schemes existing in the literature.

HierFedPDP:Hierarchical federated learning with personalized differential privacy