Asynchronous Multi-Model Dynamic Federated Learning Over Wireless Networks: Theory, Modeling, and Optimization

Federated learning enables privacy-preserving and data-efficient dimension prediction and part qualification across additive manufacturing factories

The future 6G network will be able to connect massive and various unmanned vehicles (UxVs) into the network, bringing novel requirements toward UxV network security and data privacy. Deploying machine learning (ML)-based intrusion detection on UxVs can be one promising approach. The conventional approach cannot fulfill security and privacy requirements, as the structure can be more flexible, and there may be a lack of a central node in the 6G-based UxV network. Moreover, UxV clients can exhibit high heterogeneity with extremely class-imbalanced training samples. With the presence of federated learning (FL), UxVs can collaborate to train and update ML models while preserving data privacy in the UxV security critical scenario. However, most past works focused on centralized approaches, using cloud servers to assist intrusion detection and service deployments. This article introduces an FL framework with a decentralized design for training ML models on UxVs using intrusion detection as a study case. This scheme can be more flexible, as UxV clients can collaborate to train and synchronize models without the central server. Simulation experiments were conducted to evaluate the efficiency of the proposed approach. Experimental results and theoretical analysis confirm that the proposed method outperforms baseline models with FedAvg and local-trained client models.

Machine learning (ML) and Deep learning (DL) models are popular in many areas, from business, medicine, industries, healthcare, transportation, smart cities, and many more. However, the conventional centralized training techniques may not apply to upcoming distributed applications, which require high accuracy and quick response time. It is mainly due to limited storage and performance bottleneck problems on the centralized servers during the execution of various ML and DL-based models. However, federated learning (FL) is a developing approach to training ML models in a collaborative and distributed manner. It allows the full potential exploitation of these models with unlimited data and distributed computing power. In FL, edge computing devices collaborate to train a global model on their private data and computational power without sharing their private data on the network, thereby offering privacy preservation by default. But the distributed nature of FL faces various challenges related to data heterogeneity, client mobility, scalability, and seamless data aggregation. Moreover, the communication channels, clients, and central servers are also vulnerable to attacks which may give various security threats. Thus, a structured vulnerability and risk assessment are needed to deploy FL successfully in real-life scenarios. Furthermore, the scope of FL is expanding in terms of its application areas, with each area facing different threats. In this paper, we analyze various vulnerabilities present in the FL environment and design a literature survey of possible threats from the perspective of different application areas. Also, we review the most recent defensive algorithms and strategies used to guard against security and privacy threats in those areas. For a systematic coverage of the topic, we considered various applications under four main categories: space, air, ground, and underwater communications. We also compared the proposed methodologies regarding the underlying approach, base model, datasets, evaluation matrices, and achievements. Lastly, various approaches’ future directions and existing drawbacks are discussed in detail.

Aim: The majority of machine learning (ML) models in healthcare are built on retrospective data, much of which is collected without explicit patient consent for use in artificial intelligence (AI) and ML applications. The primary aim of this study was to evaluate whether clinicians and scientific researchers themselves consent to provide their own data for the training of ML models. Materials and Methods: This survey was conducted through an anonymous online survey, utilizing platforms such as Telegram, LinkedIn, and Viber. The target audience comprised specific international groups, primarily Russian, German, and English-speaking, of clinicians and scientific researchers. These participants ranged in their levels of expertise and experience, from beginners to veterans. The survey centered on a singular, pivotal question: “Do You Consent to the Use of Your Biological and Private Data for Training Machine Learning and AI Models?” Respondents had the option to choose from three responses: “Yes” and “No”. Results: The survey was conducted in January 2024. A total of 119 unique and verified individuals participated in the survey. The results revealed that only 50% of respondents (63 persons) expressed consent to provide their own data for the training of ML and AI models. Conclusion: In the development of ML and AI models, particularly open-source ones, it is crucial to ascertain whether participants are willing to provide their private data. While ML algorithms can transform the nature of data, it is important to remember that the primary owner of this data is the individual. Our findings show that in 50% of the cases, even participants from scientific research and clinical backgrounds – individuals typically accountable for ensuring data quality in AI and ML model development – do not consent to the use of their data in AI and ML settings. This highlights the need for more stringent consent processes and ethical considerations in the utilization of personal data in AI and ML research.

Federated learning (FL) has been vigorously promoted in wireless edge networks as it fosters collaborative training of machine learning (ML) models while preserving individual user privacy and data security. In conventional FL, user equipment (UE) and an aggregator can collaboratively train shared global ML models by transmitting interactive ML models. In wireless edge networks, heterogeneity of multi-dimensional resources (e.g., computing and communication resources) used to train and transmit FL models may introduce stragglers, characterized by a slow update and/or transmission of local models. The stragglers can significantly degrade learning performance of FL, as the slowest participating UE can dramatically slow down the entire convergence. In this paper, to alleviate the negative impact of stragglers, we propose a dynamic straggler-aware clustering based FL (FeDSC) mechanism via adaptive UE clustering. Specifically, we first group participating UEs into multiple clusters based on their available computing and wireless resources. Then, we propose an adaptive clustering scheme to synchronously update the cluster aggregation model. Meanwhile, an edge server performs global aggregation of different cluster models in an asynchronous manner. Finally, we theoretically demonstrate the convergence of our proposed mechanism via numerical results. Numerical results show that our proposed mechanism can effectively reduce training time and wireless bandwidth consumption, while improving training efficiency and guaranteeing learning accuracy.

In recent years, advances in diagnosing and classifying diseases using machine learning (ML) have grown exponentially. However, due to the many privacy regulations regarding personal data, pooling together data from multiple sources and storing them in a single (centralized) location for traditional ML model training are often infeasible. Federated learning (FL), a collaborative learning paradigm, can sidestep this major pitfall by creating a global ML model that is trained by aggregating model weights from individual models that are separately trained on their own data silos, therefore avoiding most data privacy concerns. This study addresses the centralized data issue with FL by applying a novel DataWeightedFed architectural approach for effective fundus disease diagnosis from ophthalmic images. It includes a novel method for aggregating model weights by comparing the size of each model’s data and taking a dynamically weighted average of all the model’s weights. Experimental results showed a small average 1.85% loss in accuracy when training using FL compared to centralized ML model systems, a nearly 92% improvement over the conventional 55% accuracy loss. The obtained results demonstrate that this study’s FL architecture can maximize both privacy preservation and accuracy for ML in fundus disease diagnosis and provide a secure, collaborative ML model training solution within the eye healthcare space.

The exponentially growing Internet of Things (IoT) devices produce gigantic amounts of data every day. To process this ample IoT data, traditional centralized processing methods are mostly used, which face significant latency, bandwidth, and privacy challenges. Federated Learning (FL), a distributed machine learning (ML) technique, has become a promising solution to address these challenges by integrating edge computing environments. Federated edge intelligence can improve model accuracy and data privacy, and reduce latency and bandwidth. This approach allows IoT networks to train ML models collaboratively without centralizing data. This improves efficiency and preserves privacy and security. This chapter presents the fundamentals of FL, its architectural aspects, and demonstrates how it helps to further improve edge intelligence in IoT environments. We also discuss how edge devices can learn and train models with the use of the FL method. Furthermore, we provide examples of practical application use case scenarios, showcasing how this framework enhances real-time decision-making capabilities even while addressing data privacy. Finally, our research aims to guide future scholars through identifying the challenges this architecture currently faces and the future research directions associated with FL-based edge intelligence for IoT.

Conventional machine learning (ML) needs centralized training data to be present on a given machine or datacenter. The healthcare, finance, and other institutions where data sharing is prohibited require an approach for training ML models in secured architecture. Recently, techniques such as federated learning (FL), MIT Media Lab's Split Neural networks, blockchain, aim to address privacy and regulation of data. However, there are difference between the design principles of FL and the requirements of Institutions like healthcare, finance, etc., which needs blockchain-orchestrated FL having the following features: clients with their local data can define access policies to their data and define how updated weights are to be encrypted between the workers and the aggregator using blockchain technology and also prepares audit trail logs undertaken within network and it keeps actual list of participants hidden. This is expected to remove barriers in a range of sectors including healthcare, finance, security, logistics, governance, operations, and manufacturing.

Batteries, recognized as effective energy storage solutions, are considered the main facilitators of the world-wide transition towards clean and renewable energy sources. Among different types of batteries, lithium-ion (Li-ion) variants offer higher energy densities and relatively longer life spans when compared to other types. Nonetheless, a primary concern with these batteries is their lifetime. Batteries undergo various degradation mechanisms under storage and use, significantly impacting their lifespan. To this end, it is crucial to predict the degradation and lifetime of Li-ion batteries under given conditions.Researchers use three main methodologies to perform battery health diagnostics and to predict the lifetime of batteries. One approach revolves around using mechanistic, first-principle electrochemical models, also known as physics-based models. If equipped with proper thermodynamic theories, such models can show promising capabilities; however, holistic degradation prediction with these models is still challenging due to the computational complexities and the multitude of parameters that need to be fine-tuned in this approach. The other common strategy is to use empirical models to predict battery degradation. These models generally entail fewer parameters to be identified and are computationally less intensive to solve. Nonetheless, empirical models can suffer from the accuracy point-of-view as they are constrained to predict degradation trends introduced by certain degradation modes. Another common technique in battery life prediction is to utilize purely data-driven methods, such as machine learning (ML) algorithms, which also have shown promising results in the literature on rapid health predictions. However, these methods require large volumes of experimental data for training and testing ML models to ensure accuracy. In addition, data-driven methods are likely to extrapolate poorly to conditions beyond their training data and are indifferent towards the underlying degradation mechanisms. Recently, physics-informed machine learning (PI-ML) methods have garnered significant attention. They integrate physics-based or empirical models (developed based on physics) with a data-driven approach and allow one to train ML models on a smaller set of experimental data.To the best of the authors’ knowledge, the performance comparison between first-principle and empirical models when integrated within PI-ML remains unclear. Therefore, in this work, we aim to compare these two models when applied to prognostics (capacity forecasting and remaining useful life prediction) of a set of Li-ion batteries. To perform this study, we generate aging data for 40 Li-ion coin cells cycled under randomized conditions. Each cell undergoes a three-step charging stage followed by a two-step discharge stage. After obtaining the aging data, we will develop two PI-ML models, one equipped with a physics-based model and another with a set of empirical models. Both PI-ML models in this work will follow the sequential integration approach, where the training data for the final PI-ML model come from both experimental and computational data, the latter of which are obtained from the physics-based or empirical models. The parameters for the physics-based and empirical models are identified from another set of experimental data. Finally, the PI-ML models will be tested with experimental data obtained at different cycling conditions. The data flow for the sequential architecture of PI-ML is shown in the attached figure. This comparative study will help identify the performance of physics-based and empirical models when integrated into PI-ML. The main performance metric considered in this work is each model’s ability to extrapolate beyond the experimental training data set, hence aiding the final PI-ML model in generalizing to conditions not covered by its experimental training data set. Figure 1

Data privacy and regulations prevent the free transmission and sharing of information to a central place. While these regulations aim to ensure data owners can maintain clear control of their data, they also inhibit the training of machine learning (ML) algorithms and the analysis of processes that would benefit multiple stakeholders. Federated learning (FL) has provided an alternative to ensure multiple data owners collaboratively train ML models without sharing their data with each other. Sharing knowledge, rather than data, has allowed the mitigation of some of data privacy exposure risks. However, there are multiple open challenges that need to be addressed. In this talk, I will provide an overview of the state of the art and open challenges. Some of the topics I will discuss include the impact of FL on inference of private data, fairness, and transparency.

In this paper, the optimization of network performance to support the deployment of federated learning (FL) is investigated. In particular, in the considered model, each user owns a machine learning (ML) model by training through its own dataset, and then transmits its ML parameters to a base station (BS) which aggregates the ML parameters to obtain a global ML model and transmits it to each user. Due to limited radio frequency (RF) resources, the number of users that participate in FL is restricted. Meanwhile, each user uploading and downloading the FL parameters may increase communication costs thus reducing the number of participating users. To this end, we propose to introduce visible light communication (VLC) as a supplement to RF and use compression methods to reduce the resources needed to transmit FL parameters over wireless links so as to further improve the communication efficiency and simultaneously optimize wireless network through user selection and resource allocation. This user selection and bandwidth allocation problem is formulated as an optimization problem whose goal is to minimize the training loss of FL. We first use a model compression method to reduce the size of FL model parameters that are transmitted over wireless links. Then, the optimization problem is separated into two subproblems. The first subproblem is a user selection problem with a given bandwidth allocation, which is solved by a traversal algorithm. The second subproblem is a bandwidth allocation problem with a given user selection, which is solved by a numerical method. The ultimate user selection and bandwidth allocation are obtained by iteratively compressing the model and solving these two subproblems. Simulation results show that the proposed FL algorithm can improve the accuracy of object recognition by up to 16.7% and improve the number of selected users by up to 68.7%, compared to a conventional FL algorithm using only RF.

Accurate estimation of reference evapotranspiration (ETo) plays a vital role in irrigation and water resource planning. The Penman–Monteith method recommended by the Food and Agriculture Organization (FAO PM56) is widely used and considered a standard to calculate ETo. However, FAO PM56 cannot be used with limited meteorological variables, so it is compulsory to choose an alternative model for ETo estimation, which requires fewer variables. This study built ten machine learning (ML) models based on multi-function, neural network, and tree-based structure against the FAO PM56 method. For this purpose, time series temperature data on a monthly scale are only used to train ML models. The developed ML models were applied to estimate ETo at different test stations and the obtained results were compared with the FAO PM56 method to verify and validate their performance in ETo estimation for the selected stations. In addition, multiple statistical indicators, including root-mean-square error (RMSE), coefficient of determination (R2), mean absolute error (MAE), Nash–Sutcliffe efficiency (NSE), and correlation coefficient (r) were calculated to compare the performance of each ML model on ETo estimation. Among the applied ML models, the ETo tree boost (TB) ML model outperformed the other ML models in estimating ETo in diverse climatic conditions based on statistical indicators (R2, NSE, r, RMSE, and MAE). Moreover, the observed R2, NSE, and r were the highest for the TB ML model, while RMSE and MAE were found to be the lowest at the study sites compared to other applied ML models. Lastly, ETo point data yielded from the TB ML model was used in an interpolation process to create monthly and annual ETo maps. Based on the ETo maps, this study suggests mainly a focus on areas with high ETo values and proper irrigation scheduling of crops to ensure water sustainability.

Smart energy performance monitoring and optimisation at the supplier and consumer levels is essential to realising smart cities. In order to implement a more sustainable energy management plan, it is crucial to conduct a better energy forecast. The next-generation smart meters can also be used to measure, record, and report energy consumption data, which can be used to train machine learning (ML) models for predicting energy needs. However, sharing energy consumption information to perform centralised learning may compromise data privacy and make it vulnerable to misuse, in addition to incurring high transmission overhead on communication resources. This study addresses these issues by utilising federated learning (FL), an emerging technique that performs ML model training at the user/substation level, where data resides. We introduce FedraTrees, a new, lightweight FL framework that benefits from the outstanding features of ensemble learning. Furthermore, we developed a delta-based FL stopping algorithm to monitor FL training and stop it when it does not need to continue. The simulation results demonstrate that FedraTrees outperforms the most popular federated averaging (FedAvg) framework and the baseline Persistence model for providing accurate energy forecasting patterns while taking only 2% of the computation time and 13% of the communication rounds compared to FedAvg, saving considerable amounts of computation and communication resources.

With increasing concerns for data privacy and ownership, recent years have witnessed a paradigm shift in machine learning (ML). An emerging paradigm, federated learning (FL), has gained great attention and has become a novel design for machine learning implementations. FL enables the ML model training at data silos under the coordination of a central server, eliminating communication overhead and without sharing raw data. In this paper, we conduct a review of the FL paradigm and, in particular, compare the types, the network structures, and the global model aggregation methods. Then, we conducted a comprehensive review of FL applications in the energy domain (refer to the smart grid in this paper). We provide a thematic classification of FL to address a variety of energy-related problems, including demand response, identification, prediction, and federated optimizations. We describe the taxonomy in detail and conclude with a discussion of various aspects, including challenges, opportunities, and limitations in its energy informatics applications, such as energy system modeling and design, privacy, and evolution.

Reconstructing the emergence of the modern continental crust is crucial to understand the evolution of the crust, the onset of plate tectonics, elemental cycling, and long-term climate. However, it remains highly contentious about when and how the major subaerial continental crust emerged over time. Here, we used a machine learning (ML) model (XGBoost) to reveal the exposed history of continental crust over the last 3.8 billion years ago (Ga). First, we compiled ~10,000 modern subaerial or submarine basalts with major and trace elements to train the ML model. Then, the trained ML model (with resampling) was utilized to predict and calculate the mean proportions of subaerially erupted continental basaltic rocks since 3.8 Ga. The result suggested that the subaerial proportion only reached about 50% at ~2.5 Ga, indicating the exposure of the continental crust was far from the present-day level at the end of the Archean era. On the other hand, since ~1.8 Ga, the subaerial proportion of the continental crust exhibited a dynamic balance between ~60% and 80%, reaching the present-day level.