Secure and Efficient Federated Learning With Provable Performance Guarantees via Stochastic Quantization

Normalization is an important but understudied challenge in privacy-related application domains such as federated learning (FL), differential privacy (DP), and differentially private federated learning (DP-FL). While the unsuitability of batch normalization for these domains has already been shown, the impact of other normalization methods on the performance of federated or differentially private models is not well-known. To address this, we draw a performance comparison among layer normalization (LayerNorm), group normalization (GroupNorm), and the recently proposed kernel normalization (KernelNorm) in FL, DP, and DP-FL settings. Our results indicate LayerNorm and GroupNorm provide no performance gain compared to the baseline (i.e. no normalization) for shallow models in FL and DP. They, on the other hand, considerably enhance the performance of shallow models in DP-FL and deeper models in FL and DP. KernelNorm, moreover, significantly outperforms its competitors in terms of accuracy and convergence rate (or communication efficiency) for both shallow and deeper models in all considered learning environments. Given these key observations, we propose a kernel normalized ResNet architecture called KNResNet-13 for differentially private learning. Using the proposed architecture, we provide new state-of-the-art accuracy values on the CIFAR-10 and Imagenette datasets, when trained from scratch.

Within the federated learning (FL) framework, the client collaboratively trains the model in coordination with a central server, while the training data can be kept locally on the client. Thus, the FL framework mitigates the privacy disclosure and costs related to conventional centralized machine learning. Nevertheless, current surveys indicate that FL still has problems in terms of communication efficiency and privacy risks. In this paper, to solve these problems, we develop an FL framework with communication-efficient and privacy-preserving (FLCP). To realize the FLCP, we design a novel compression algorithm with efficient communication, namely, adaptive weight compression FedAvg (AWC-FedAvg). On the basis of the non-independent and identically distributed (non-IID) and unbalanced data distribution in FL, a specific compression rate is provided for each client, and homomorphic encryption (HE) and differential privacy (DP) are integrated to provide demonstrable privacy protection and maintain the desirability of the model. Therefore, our proposed FLCP smoothly balances communication efficiency and privacy risks, and we prove its security against “honest-but-curious” servers and extreme collusion under the defined threat model. We evaluate the scheme by comparing it with state-of-the-art results on the MNIST and CIFAR-10 datasets. The results show that the FLCP performs better in terms of training efficiency and model accuracy than the baseline method.

HierFedPDP:Hierarchical federated learning with personalized differential privacy

Federated learning (FL) is an attractive scheme for training machine learning models across distributed edge devices without requiring raw data to be shared. However, deploying FL at the edge of the net- work raises two interrelated issues: (1) the significant communication overhead due to transmitting massive gradient vectors from resource- constrained devices and (2) the risk of privacy leakage via attacks that can invert gradients or infer training samples. Current methods treat gradient compression and differential privacy (DP) as separate concerns, leading to compounded accuracy loss. We propose AGC-DP—Adaptive Gradient Compression (AGC) with Differential Privacy (DP)—a uni- fied framework that achieves efficient communication and strong pri- vacy guarantees across diverse edge FL environments. AGC-DP also introduces a resource-aware client selection algorithm to balance con- tributions from battery level, available memory, and uplink bandwidth; an adaptive compression ratio for each client based on the optimization problem; and a privacy amplification construction that exploits sub- sampling to improve DP accounting. We provide rigorous theoretical analysis, including convergence guarantees under non-i.i.d. data dis- tributions and formal (ϵ, δ)- DP proofs. Comprehensive experiments across the CIFAR-10, HAR, and FEMNIST datasets with up to 100 edge clients show that AGC-DP reduces communication overhead by up to 91.1% compared to vanilla FedAvg, while maintaining accuracy within 1.5% of the vanilla baseline. Our results show that privacy, com- munication efficiency, and model utility can be achieved simultaneously in resource-constrained edge FL.

In recent years, to surmount the challenges of data sensitivity and data silos, federated learning (FL), a collaborative decentralized privacy preservation machine learning framework was proposed and rapidly became a current research hotspot. In this paper, we present an overview of FL workflow, as well as the communication overhead and privacy leakage problems associated with FL. In order to alleviate these two pain points, several corresponding solutions are presented in this paper along with their advantages and disadvantages. To reduce the communication overhead, this paper suggests three reliable approaches by reviewing related research including increasing the amount of computing on the edge side, model compression, and improving parallelism. To address the privacy concerns, this paper introduces three prevailing privacy preservation techniques: differential privacy, homomorphic encryption, and secure multiparty computation (SMC), and shows how to efficiently incorporate these privacy-preserving techniques into a federated learning framework by summarizing relevant literature. On the whole, this paper contributes to explaining the overall workflow of a FL framework and helps advance the understanding of FL in improving communication efficiency and enhancing privacy security.

Privacy-preserving Federated Learning (FL) based on Differential Privacy (DP) protects clients’ data by adding DP noise to samples’ gradients and has emerged as a de facto standard for data privacy in FL. However, the accuracy of global models in DP-based FL may be reduced significantly when rogue clients occur who deviate from the preset DP-based FL approaches and selfishly inject excessive DP noise beyond expectations, thereby applying a smaller privacy budget in the DP mechanism to ensure a higher level of security. Existing DP-based FL fails to prevent such attacks as they are imperceptible. Under the DP-based FL system and random Gaussian noise, the local model parameters of the rogue clients and the honest clients have identical distributions. In particular, the rogue local models show a low performance, but directly filtering out lower-performance local models compromises the generalizability of global models, as local models trained on scarce data also behave with low performance in the early epoch. In this paper, we propose ReFL, a novel privacy-preserving FL system that enforces DP and avoids the accuracy reduction of global models caused by excessive DP noise of rogue clients. Based on the observation that rogue local models with excessive DP noise and honest local models trained on scarce data have different performance patterns in long-term training epochs, we propose a long-term contribution incentives scheme to evaluate clients’ reputations and identify rogue clients. Furthermore, we design a reputation-based aggregation to avoid the damage of rogue clients’ models on the global model accuracy, based on the incentive reputation. Extensive experiments demonstrate ReFL guarantees the global model accuracy performance 0.77% - 81.71% higher than existing DP-based FL methods in the presence of rogue clients.

The convergence of federated learning (FL) and differential privacy (DP) presents a transformative approach to secure predictive modeling in healthcare finance, where safeguarding sensitive patient and financial data is paramount. Traditional centralized machine learning methods often raise significant privacy concerns due to the necessity of aggregating data from multiple institutions. Federated learning mitigates this by enabling decentralized model training across disparate data sources, such as hospitals, insurance firms, and financial institutions, without exposing raw data. However, FL alone remains vulnerable to inference and reconstruction attacks. To enhance security, differential privacy introduces mathematically rigorous noise mechanisms that obfuscate sensitive information while preserving data utility. This paper explores the synergistic integration of DP and FL for building robust, privacy-preserving predictive models tailored to healthcare finance applications, such as fraud detection, insurance risk scoring, billing optimization, and cost forecasting. We discuss the architectural design, privacy-utility trade-offs, and implementation challenges involved, including issues of scalability, model accuracy, regulatory compliance (e.g., HIPAA and GDPR), and communication overhead. Furthermore, real-world use cases and simulation results demonstrate the efficacy of DP-FL frameworks in delivering secure and accurate predictive insights without compromising individual or institutional privacy. The study concludes by highlighting open research directions and recommending best practices for deploying privacy-enhanced federated learning systems in complex, multi-stakeholder healthcare financial ecosystems.

The current AI revolution originated from the impressive deep learning methodology. Deep neural networks and in particular recent foundation models rely heavily on vast amounts of data and computing power, which poses the question of how to efficiently train large AI models from big data distributed geographically in multiple organisations. Because of privacy concerns and commercial interest of the data providers, as well as mandatory regulation rules such as General Data Protection Regulation (GDPR) of the European Union, sensitive local data must not be leaked to others. This restriction makes traditional distributed learning paradigms an unfeasible solution in which local information can freely flow among nodes. Federated learning is an emerging distributed learning paradigm that features privacy protection by utilising sophisticated algorithms such as differential privacy and aggregation schemes to generate global results. However, while federated learning has made great progress and been deployed in many industrial systems, there remain obstacles in the canonical paradigm that prevent its advance in the era of large models. In this thesis, two important difficulties are identified: low training efficiency and lack of explainability. Through experimental results from the literature and collected operating stat- istics of deployed federated systems, the main training bottleneck boils down to the transfer time of large messages, which could add up to more than one day in a typical federated training across distant sites. On the other hand, explainability can help evaluate the quality of training samples, shorten training time, and facilitate the acceptance of complex AI models. These advantages are valuable in federated environments because there is no authoritative control over the data quantity of participants. Motivated by the aforementioned challenges of large-scale federated learning, the topic of this thesis is efficient and explainable large-scale federated learning. An extensive investigation of the literature and our works on the topic are presented. Specifically, our contributions are as follows: • We propose Hy pergradient for Data Relevance Analysis (HyDRA), which is an explainable approach that interprets the predictions made by deep neural networks as effects of training samples. Existing approaches like Influence Function generally estimate training sample contributions around the final model parameters and ignore how the training samples shape the optimisation trajectory. In contrast, HyDRA assesses the contribution of training samples to test samples throughout the optimisation trajectory. In order to accelerate computation, we provide an approximation formula and prove that, under moderate conditions, the approximation error is bounded. Consistent with the theoretical claim, empirical results indicate that the error is indeed small. In addition, we quantitatively demonstrate that HyDRA outperforms Influence Function in accurately estimating data contribution and detecting noisy data labels. • We propose the Guided T runcation G radient Shapley (GTG-Shapley) ap- proach to fairly evaluate the contributions of the participants to the perform- ance of the final model without exposing their private data. To sustain the long-term operation of a federated learning (FL) ecosystem, it is important to attract high-quality data owners with appropriate incentive schemes. Fair evaluation of participant contributions is an important building block of such incentive schemes. Although Shapley value-based techniques have been widely adopted to provide fair evaluation in such cases, existing approaches incur a significant computational cost, making them difficult to apply in practice. GTG-Shapley approximates sub-coalitions by aggregating sub- models, rather than repeatedly training from scratch. It also uses Monte Carlo sampling and other optimisations to further reduce the number of sub- coalition evaluations. The experimental results showed that GTG-Shapley can closely approximate the actual Shapley value, while significantly increasing the computational efficiency compared to other advanced methods, especially in non-i.i.d. settings. • We propose Fed erated O pportunistic B lock D ropout (FedOBD), which provides an efficient framework for federated learning. The key novelty is that it decomposes large neural networks into semantic blocks so that federated participants can opportunistically upload quantised blocks, which can greatly speed up training. Extensive experiments evaluating FedOBD against four competitive approaches based on common datasets show that it reduces the overall communication overhead by more than 88% compared to the best-performing baseline approach, while achieving the highest test accuracy. FedOBD was deployed in industry and helped the company reduce the training communication overhead by more than 70% compared to its previous AI engine, while maintaining model performance at a test F1 score of over 85%. Finally, we discuss further research directions, including: (1) integrating the pro- posed methods into a unified pipeline that provides hierarchical explanations at both sample and participant levels while maintaining communication efficiency; (2) extending the efficient communication techniques to federated graph learning, where convolutional operations over distributed sub-graphs incur high communication costs; and (3) balancing explainability and productivity in deployed industrial AI systems.

The healthcare industry has found it challenging to build a powerful global classification model due to the scarcity and diversity of medical data. The leading cause is privacy, which restricts data sharing among healthcare providers. Federated learning (FL) can contribute to developing classification models by protecting data privacy. This study has tested various federated techniques in a peer‐to‐peer setting to classify brain Magnetic Resonance Images (MRI). The authors propose various aggregation strategies for FL, including Federated Averaging (FedAvg), Quantum FL with FedAVG (QFedAvg) and Fault Tolerant FedAvg (Ft‐FedAvg) and FedAvg with differential privacy (Dp‐FedAvg). In each approach, a custom Convolutional Neural Network (CNN) model is applied to compute locally run nodes with different parts of the same brain MRI dataset for 10, 20 and 30 training and test rounds. A central server and CNN‐based three federated clients are included in the FL‐based brain tumour classification model to exchange data and combine the model weights on the server, which are sent from local devices to the server. The superiority of the performance of the proposed model is demonstrated by comparing it with traditional methods on various performance metrics. Experimental results show that in brain MRI dataset classification using FL approaches, FedAVg showed the best performance with 85.55% and 84.60% success for 10 and 20 rounds, respectively, while Ft‐FedAvg showed the best performance with 85.80% success for 30 rounds for test set. Statistical results obtained from FL approaches showed that FedAvg and Ft‐FedAvg have superior performance with regard to accuracy and robustness in comparison with the others.

Privacy and data security have become the new hot topic for regulators in recent years. As a result, Federated Learning (FL) (also called collaborative learning) has emerged as a new training paradigm that allows multiple, geographically distributed nodes to learn a Deep Learning (DL) model together without sharing their data. Blockchain is becoming a new trend as data protection and privacy are concerns in many sectors. Technology is leading the world and transforming into a global village where everything is accessible and transparent. We have presented a blockchain enabled security model using FL that can generate an enhanced DL model without sharing data and improve privacy through higher security and access rights to data. However, existing FL approaches also have unique security vulnerabilities that malicious actors can exploit and compromise the trained model. The FL method is compared to the other known approaches. Users are more likely to choose the latter option, i.e., providing local but private data to the server and using ML apps, performing ML operations on the devices without benefiting from other users’ data, and preventing direct access to raw data and local training of ML models. FL protects data privacy and reduces data transfer overhead by storing raw data on devices and combining locally computed model updates. We have investigated the feasibility of data and model poisoning attacks under a blockchain-enabled FL system built alongside the Ethereum network and the traditional FL system (without blockchain). This work fills a knowledge gap by proposing a transparent incentive mechanism that can encourage good behavior among participating decentralized nodes and avoid common problems and provides knowledge for the FL security literature by investigating current FL systems.

Federated Learning (FL) has become one of the paradigms shifting towards machine learning, allowing a number of edge devices to jointly train models without exchanging raw data. This can be used to guarantee privacy of the data, less data communication overhead, and real-time decision-making under edge computing conditions. The paper discusses federated learning to secure edge computing, focusing on architecture, communication models, aggregation, and security. The experimental and theoretical studies show that FL is capable of reaching model accuracy that is similar to the centralized learning and reducing privacy threats and adversarial risks. The problems, including the heterogeneity of data, and the lack of computing power and communication efficiency, are discussed critically. The research paper ends with the suggestions on how to improve FL implementation in the actual edge networks, focusing on secure aggregation protocols, differential privacy, and client selection adaption strategies.

Large language models (LLMs) have made significant advances in the field of natural language processing (NLP). However, their centralized training approach faces challenges related to data privacy, communication efficiency, and robustness against adversarial attacks, particularly in wireless environments. With the gradual depletion of high-quality public data, there is an urgent need to leverage private data distributed across various parties. Although federated learning (FL) offers a privacy-preserving collaborative training paradigm, it struggles to meet the high computational demands of edge devices and remains vulnerable to adversarial attacks. This paper introduces <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ROFED</small>-LLM, a novel framework for robust, privacy-preserving training of LLMs on decentralized private data over wireless networks. By integrating split federated learning, which partitions the model across devices to enhance privacy, with adaptive jamming defense mechanisms, <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ROFED</small>-LLM enables collaborative LLM training without raw data sharing while ensuring resilience against wireless adversarial attacks. Our multi-modal defense strategy combines model-level protections, such as differential privacy and dynamic pruning, with communication-level safeguards, including adaptive beamforming which optimizes wireless signal transmission to mitigate interference, and resource allocation optimization. Extensive experiments across diverse NLP tasks demonstrate <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ROFED</small>-LLM's superiority, achieving a 12.87% improvement in privacy preservation and 18.26% enhancement in jamming resilience compared to existing methods such as FedAvg and SCAFFOLD, with only a marginal 3.94% trade-off in model accuracy. Our code repository has been open sourced at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://anonymous.4open.science/r/RoFed-LLM-54E1</uri>.

In domains such as remote healthcare, where sensitive data must be protected, Federated Learning (FL) has emerged as a radical new way to build privacy-preserving AI models. In contrast, centralized AI systems have always existed, aggregating patient data into a single repository, subject to vulnerabilities in privacy, security, and regulatory compliance. In this study, we propose a novel FL implementation in remote healthcare that supports distributed training across multiple devices and remote healthcare facilities, while preserving patient privacy. We design a hypothetical framework taking advantage of the state of the art in machine learning tools including TensorFlow Federated and privacy enhancing technologies like differential privacy and secure aggregation. We evaluate the effectiveness of FL for simulating patient vitals, symptoms, and outcomes for different healthcare institutions in the research. This shows that FL can get comparable model accuracy with centralized systems, and its privacy and scalability aspect gains much more. Communication overhead and data heterogeneity are discussed, and practical strategies around their mitigation are laid out for the practical deployment of this method. In this work, we offer a comprehensive analysis of the application of FL in the healthcare domain, particularly on how it can contribute to the security of FL-sensitive data and the development of medical AI applications. Future research direction in building privacy-preserving AI models specific to the fluctuating needs of remote healthcare environments is now feasible with these results.

The rapid evolution of cyber threats necessitates innovative approaches to enhance global cybersecurity collaboration. Federated Learning (FL) has emerged as a decentralized machine learning paradigm that enables distributed threat intelligence sharing while maintaining data privacy and security. This chapter explores the application of FL for large-scale cybersecurity networks, addressing critical challenges in scalability, security, and communication efficiency. The focus is on optimizing secure aggregation techniques to enable efficient and privacy-preserving model updates across heterogeneous and resource-constrained environments. Key solutions such as hierarchical aggregation, sparse model updates, and blockchain-based enhancements are discussed to mitigate the computational and communication overheads inherent in federated systems. the chapter investigates the integration of advanced cryptographic methods, including homomorphic encryption and differential privacy, to strengthen the security of federated networks against adversarial attacks. By leveraging FL’s potential, organizations can share threat intelligence across global networks without compromising sensitive data, significantly improving real-time cyber threat detection and response. The chapter concludes by identifying future research directions for overcoming existing challenges and further optimizing federated models in cybersecurity.

With the continuous improvement of hardware computing power, edge computing of industrial data has been gradually applied. In the past decade, the promotion of edge computing has also greatly improved the efficiency of industrial production. Compared with the conventional cloud computing, it not only saves the bandwidth consumption of data transmission, but also ensures the terminal data security to a certain extent. However, the continuous update of attack types also put forward new requirements for the privacy protection of industrial edge computing. So it should fundamentally solve the risk of industrial data leakage in the process of deep model training in edge terminal. In this article, we propose a new federated edge learning framework based on hybrid differential privacy and adaptive compression for industrial data processing. Specifically, it first completes the adaptive gradient compression preparation, then constructs the industrial federated learning model, and finally makes use of adaptive differential privacy model to optimize, so as to complete the privacy protection towards the transmission of gradient parameters in industrial environment. By optimizing the hybrid differential privacy and adaptive compression, we can better prevent the terminal data privacy against inference attacks. The experimental results show that this method is very effective in the industrial edge computing situation, and it also opens up a new direction for the effect of differential privacy in federated learning.