Privacy-Preserving in Machine Learning: Differential Privacy Case Study

Aim: The majority of machine learning (ML) models in healthcare are built on retrospective data, much of which is collected without explicit patient consent for use in artificial intelligence (AI) and ML applications. The primary aim of this study was to evaluate whether clinicians and scientific researchers themselves consent to provide their own data for the training of ML models. Materials and Methods: This survey was conducted through an anonymous online survey, utilizing platforms such as Telegram, LinkedIn, and Viber. The target audience comprised specific international groups, primarily Russian, German, and English-speaking, of clinicians and scientific researchers. These participants ranged in their levels of expertise and experience, from beginners to veterans. The survey centered on a singular, pivotal question: “Do You Consent to the Use of Your Biological and Private Data for Training Machine Learning and AI Models?” Respondents had the option to choose from three responses: “Yes” and “No”. Results: The survey was conducted in January 2024. A total of 119 unique and verified individuals participated in the survey. The results revealed that only 50% of respondents (63 persons) expressed consent to provide their own data for the training of ML and AI models. Conclusion: In the development of ML and AI models, particularly open-source ones, it is crucial to ascertain whether participants are willing to provide their private data. While ML algorithms can transform the nature of data, it is important to remember that the primary owner of this data is the individual. Our findings show that in 50% of the cases, even participants from scientific research and clinical backgrounds – individuals typically accountable for ensuring data quality in AI and ML model development – do not consent to the use of their data in AI and ML settings. This highlights the need for more stringent consent processes and ethical considerations in the utilization of personal data in AI and ML research.

Applying machine learning (ML) to sensitive domains requires privacy protection of the underlying training data through formal privacy frameworks, such as differential privacy (DP). Yet, usually, the privacy of the training data comes at the cost of the resulting ML models' utility. One reason for this is that DP uses one uniform privacy budget epsilon for all training data points, which has to align with the strictest privacy requirement encountered among all data holders. In practice, different data holders have different privacy requirements and data points of data holders with lower requirements can contribute more information to the training process of the ML models. To account for this need, we propose two novel methods based on the Private Aggregation of Teacher Ensembles (PATE) framework to support the training of ML models with individualized privacy guarantees. We formally describe the methods, provide a theoretical analysis of their privacy bounds, and experimentally evaluate their effect on the final model's utility using the MNIST, SVHN, and Adult income datasets. Our empirical results show that the individualized privacy methods yield ML models of higher accuracy than the non-individualized baseline. Thereby, we improve the privacy-utility trade-off in scenarios in which different data holders consent to contribute their sensitive data at different individual privacy levels.

The future 6G network will be able to connect massive and various unmanned vehicles (UxVs) into the network, bringing novel requirements toward UxV network security and data privacy. Deploying machine learning (ML)-based intrusion detection on UxVs can be one promising approach. The conventional approach cannot fulfill security and privacy requirements, as the structure can be more flexible, and there may be a lack of a central node in the 6G-based UxV network. Moreover, UxV clients can exhibit high heterogeneity with extremely class-imbalanced training samples. With the presence of federated learning (FL), UxVs can collaborate to train and update ML models while preserving data privacy in the UxV security critical scenario. However, most past works focused on centralized approaches, using cloud servers to assist intrusion detection and service deployments. This article introduces an FL framework with a decentralized design for training ML models on UxVs using intrusion detection as a study case. This scheme can be more flexible, as UxV clients can collaborate to train and synchronize models without the central server. Simulation experiments were conducted to evaluate the efficiency of the proposed approach. Experimental results and theoretical analysis confirm that the proposed method outperforms baseline models with FedAvg and local-trained client models.

Batteries, recognized as effective energy storage solutions, are considered the main facilitators of the world-wide transition towards clean and renewable energy sources. Among different types of batteries, lithium-ion (Li-ion) variants offer higher energy densities and relatively longer life spans when compared to other types. Nonetheless, a primary concern with these batteries is their lifetime. Batteries undergo various degradation mechanisms under storage and use, significantly impacting their lifespan. To this end, it is crucial to predict the degradation and lifetime of Li-ion batteries under given conditions.Researchers use three main methodologies to perform battery health diagnostics and to predict the lifetime of batteries. One approach revolves around using mechanistic, first-principle electrochemical models, also known as physics-based models. If equipped with proper thermodynamic theories, such models can show promising capabilities; however, holistic degradation prediction with these models is still challenging due to the computational complexities and the multitude of parameters that need to be fine-tuned in this approach. The other common strategy is to use empirical models to predict battery degradation. These models generally entail fewer parameters to be identified and are computationally less intensive to solve. Nonetheless, empirical models can suffer from the accuracy point-of-view as they are constrained to predict degradation trends introduced by certain degradation modes. Another common technique in battery life prediction is to utilize purely data-driven methods, such as machine learning (ML) algorithms, which also have shown promising results in the literature on rapid health predictions. However, these methods require large volumes of experimental data for training and testing ML models to ensure accuracy. In addition, data-driven methods are likely to extrapolate poorly to conditions beyond their training data and are indifferent towards the underlying degradation mechanisms. Recently, physics-informed machine learning (PI-ML) methods have garnered significant attention. They integrate physics-based or empirical models (developed based on physics) with a data-driven approach and allow one to train ML models on a smaller set of experimental data.To the best of the authors’ knowledge, the performance comparison between first-principle and empirical models when integrated within PI-ML remains unclear. Therefore, in this work, we aim to compare these two models when applied to prognostics (capacity forecasting and remaining useful life prediction) of a set of Li-ion batteries. To perform this study, we generate aging data for 40 Li-ion coin cells cycled under randomized conditions. Each cell undergoes a three-step charging stage followed by a two-step discharge stage. After obtaining the aging data, we will develop two PI-ML models, one equipped with a physics-based model and another with a set of empirical models. Both PI-ML models in this work will follow the sequential integration approach, where the training data for the final PI-ML model come from both experimental and computational data, the latter of which are obtained from the physics-based or empirical models. The parameters for the physics-based and empirical models are identified from another set of experimental data. Finally, the PI-ML models will be tested with experimental data obtained at different cycling conditions. The data flow for the sequential architecture of PI-ML is shown in the attached figure. This comparative study will help identify the performance of physics-based and empirical models when integrated into PI-ML. The main performance metric considered in this work is each model’s ability to extrapolate beyond the experimental training data set, hence aiding the final PI-ML model in generalizing to conditions not covered by its experimental training data set. Figure 1

Reconstructing the emergence of the modern continental crust is crucial to understand the evolution of the crust, the onset of plate tectonics, elemental cycling, and long-term climate. However, it remains highly contentious about when and how the major subaerial continental crust emerged over time. Here, we used a machine learning (ML) model (XGBoost) to reveal the exposed history of continental crust over the last 3.8 billion years ago (Ga). First, we compiled ~10,000 modern subaerial or submarine basalts with major and trace elements to train the ML model. Then, the trained ML model (with resampling) was utilized to predict and calculate the mean proportions of subaerially erupted continental basaltic rocks since 3.8 Ga. The result suggested that the subaerial proportion only reached about 50% at ~2.5 Ga, indicating the exposure of the continental crust was far from the present-day level at the end of the Archean era. On the other hand, since ~1.8 Ga, the subaerial proportion of the continental crust exhibited a dynamic balance between ~60% and 80%, reaching the present-day level.

The rising machine learning (ML) models have become the preferred way for landslide detection based on remote sensing images, but the performance of these models in a sample-free area are rarely concerned in many studies. In this study, we used a cross-validation method (training model in one area and validation in another) to compare the model portability of trained ML models applied in an “off-site” area, as a consideration of the landslide detection ability of these models in sample-free areas. We integrate nighttime light imagery, multi-seasonal optical Landsat time-series and digital elevation data, and we employed support vector machines (SVM), artificial neural networks (ANN) and random forest (RF) models to classify the satellite imagery and identify landslides. Samples of two scenarios generated from two subareas of the Jiuzhaigou disaster-stricken region are used for the cross-application and accuracy evaluation of three ML models. The results revealed that when the trained models are applied in areas outside those in which they were developed, the landslide identification accuracy of these three models has declined. Especially for the SVM and ANN models, the accuracy is greatly reduced and there appears a seriously imbalanced user’s and producer’s accuracy. However, although the performance of the RF model is lower than that of SVM and ANN models in their local area, the RF model exhibits stable portability, and retains the original performance and achieves a satisfactory balance between overestimation and underestimation in “off-site” areas. An additional validation from a new area proved that the landslide detection performance of the RF model with stable portability is higher than that of the SVM and ANN models in “off-site” areas. The results suggest that evaluating the model portability through cross-application can be a useful way to determine the most suitable model for landslide detection in “off-site” areas with a similar geographic environment to model development areas, so as to maximize the accuracy of landslide detection based on limited samples.

Surrogate-assisted hydraulic fracture optimization workflow with applications for shale gas reservoir development: a comparative study of machine learning models

Covalent inhibition offers many advantages over non-covalent inhibition, but covalent warhead reactivity must be carefully balanced to maintain potency while avoiding unwanted side effects. While warhead reactivities are commonly measured with assays, a computational model to predict warhead reactivities could be useful for several aspects of the covalent inhibitor design process. Studies have shown correlations between covalent warhead reactivities and quantum mechanic (QM) properties that describe important aspects of the covalent reaction mechanism. However, the models from these studies are often linear regression equations and can have limitations associated with their usage. Applications of machine learning (ML) models to predict covalent warhead reactivities with QM descriptors are not extensively seen in the literature. This study uses QM descriptors, calculated at different levels of theory, to train ML models to predict reactivities of covalent acrylamide warheads. The QM/ML models are compared with linear regression models built upon the same QM descriptors and with ML models trained on structure-based features like Morgan fingerprints and RDKit descriptors. Experiments show that the QM/ML models outperform the linear regression models and the structure-based ML models, and literature test sets demonstrate the power of the QM/ML models to predict reactivities of unseen acrylamide warhead scaffolds. Ultimately, these QM/ML models are effective, computationally feasible tools that can expedite the design of new covalent inhibitors.

Growing concerns about centralized mining of personal data threatens to stifle further proliferation of machine learning (ML) applications. Consequently, a recent trend in ML training advocates for a paradigm shift – moving the computation of ML models from a centralized server to a federation of edge devices owned by the users whose data is to be mined. Though such decentralization aims to alleviate concerns related to raw data sharing, it introduces a set of challenges due to the hardware heterogeneity among the devices possessing the data. The heterogeneity may, in the most extreme cases, impede the participation of low-end devices in the training or even prevent the deployment of the ML model to such devices. Recent research in distributed collaborative machine learning (DCML) promises to address the issue of ML model training over heterogeneous devices. However, the actual extent to which the issue is solved remains unclear, especially as an independent investigation of the proposed methods’ performance in realistic settings is missing. In this paper, we present a detailed survey and an evaluation of algorithms that aim to enable collaborative model training across diverse devices. We explore approaches that harness three major strategies for DCML, namely Knowledge Distillation, Split Learning, and Partial Training, and we conduct a thorough experimental evaluation of these approaches on a real-world testbed of 14 heterogeneous devices. Our analysis compares algorithms based on the resulting model accuracy, memory consumption, CPU utilization, network activity, and other relevant metrics, and provides guidelines for practitioners as well as pointers for future research in DCML.

Accurate estimation of reference evapotranspiration (ETo) plays a vital role in irrigation and water resource planning. The Penman–Monteith method recommended by the Food and Agriculture Organization (FAO PM56) is widely used and considered a standard to calculate ETo. However, FAO PM56 cannot be used with limited meteorological variables, so it is compulsory to choose an alternative model for ETo estimation, which requires fewer variables. This study built ten machine learning (ML) models based on multi-function, neural network, and tree-based structure against the FAO PM56 method. For this purpose, time series temperature data on a monthly scale are only used to train ML models. The developed ML models were applied to estimate ETo at different test stations and the obtained results were compared with the FAO PM56 method to verify and validate their performance in ETo estimation for the selected stations. In addition, multiple statistical indicators, including root-mean-square error (RMSE), coefficient of determination (R2), mean absolute error (MAE), Nash–Sutcliffe efficiency (NSE), and correlation coefficient (r) were calculated to compare the performance of each ML model on ETo estimation. Among the applied ML models, the ETo tree boost (TB) ML model outperformed the other ML models in estimating ETo in diverse climatic conditions based on statistical indicators (R2, NSE, r, RMSE, and MAE). Moreover, the observed R2, NSE, and r were the highest for the TB ML model, while RMSE and MAE were found to be the lowest at the study sites compared to other applied ML models. Lastly, ETo point data yielded from the TB ML model was used in an interpolation process to create monthly and annual ETo maps. Based on the ETo maps, this study suggests mainly a focus on areas with high ETo values and proper irrigation scheduling of crops to ensure water sustainability.

Empowered by promising artificial intelligence, the traditional Internet of Things is evolving into the Artificial Intelligence of Things (AIoT), which is an important enabling technology for Industry 4.0. Collaborative learning is a key technology for AIoT to build machine learning (ML) models on distributed datasets. However, there are two critical concerns of collaborative learning for AIoT: privacy leakage of sensitive data and dishonest computation. Specifically, data contains sensitive information of users, which cannot be openly shared for model learning. Furthermore, to protect the privacy of data or other selfish purposes, participants of collaborative learning may behave dishonestly, submitting dummy data or incorrect model computation. Therefore, it is important to guarantee privacy preservation of data and honest computation on collaborative learning. Our work tackles the two concerns wherein a model demander can securely train ML models with sensitive data and can regulate the computation of participants. To this end, we propose a secure and trusted collaborative learning framework called TrusCL. The framework guarantees privacy preservation via a delicate combination of homomorphic encryption (HE) and differential privacy (DP), achieving the trade-off between efficiency and accuracy. Furthermore, based on blockchain, in our design, the key steps of secure collaborative learning are recorded on blockchain so that malicious behaviors can be effectively tracked and choked in a timely manner to facilitate trusted computation. Experimental results validate the trade-off performance of Trus-CL between model training efficiency and trained model accuracy.

HADA: An automated tool for hardware dimensioning of AI applications

Federated learning enables privacy-preserving and data-efficient dimension prediction and part qualification across additive manufacturing factories

Towards Accountability in Machine Learning Applications: A System-testing Approach

Machine learning (ML) is being used to improve intrusion detection mechanisms and identification in cyber security. Network data volume scaling (with the help of Machine learning) — Automated analysis and pattern recognition for large amounts of network-data, thereby detection of anomalies / potentially malicious activities that escape current rule-based techniques. By training ML models on historical data, these models can learn benign network behavior as well the anomalies in them that may result from malicious activities. The purpose of this essay/report is to begin taking a look under the hood at how ML can be used for security threat detection and analysis in-networking on an ongoing basis. This paper covers the automation of ML algorithms to enhance network security. Given the current state of electronic threats and their evolution, traditional security methods are typically insufficient. ML can analyze large volumes of data, learn patterns from it and this makes it suitable to complement network defense mechanisms. It then discusses different ML applications in network cybersecurity: intrusion detection, anomaly detection, spam and malware analysis; which the paper characterizes. It analyzes the potential, benefits and constrains of major ML methods in network security like supervised learning; unsupervised learning and reinforcement-learning. Finally, this paper represents recent progress in the use and impact of ML techniques along with case studies.The paper discusses the existing difficulties in the field such as the necessity for datasets and the vulnerability of machine learning models to adversarial attacks.T he paper also highlights avenues for exploration by focusing on developing scalable security solutions based on machine learning that are resilient and flexible.The goal of this examination is to offer both researchers and industry professionals valuable perspectives into the opportunities and obstacles linked to utilizing machine learning, in the domain of network security. ML methods have potential to improve network security by addressing the challenges posed by the increasing cyber threats that traditional security measures struggle to combat effectively over time. One key strength of ML lies in its capacity to analyze datasets and identify intricate patterns efficiently. The research paper delves into applications of ML in enhancing network security. The list covers security tools like Intrusion Detection Systems (IDS) examining malware and phishing attempts as well as anomalies in network activity and user behavior analysis (UEBA). The study explores both supervised and unsupervised learning methods. How they are used for quick threat detection and response in real time scenarios.You will find case studies and recent developments that showcase the implementation and effectiveness of these strategies.In addition the article delves into the obstacles linked to using machine learning techniques in network security including the necessity, for datasets, The paper's goal is to give an enlightening summary to scholars and practitioners about how machine learning can be applied to network security in order to provide solutions that are robust, adaptive, and scalable. To this end, it touches on several relevant aspects. One is the threat posed by adversarial attacks on the sorts of models that are likely to be used in this context. Another is the imperative, deriving from both adversarial threat and model drift, that models needed in this context be available in a form usable for continuous update. Keywords: Network Security, machine learning (ML), Intrusion Detection Systems (IDS), entity behavior analytics (UEBA).