Optimization of machine learning model training procedure on multi-gpu systems to enhance cyber security in telecommunication networks

Aim: The majority of machine learning (ML) models in healthcare are built on retrospective data, much of which is collected without explicit patient consent for use in artificial intelligence (AI) and ML applications. The primary aim of this study was to evaluate whether clinicians and scientific researchers themselves consent to provide their own data for the training of ML models. Materials and Methods: This survey was conducted through an anonymous online survey, utilizing platforms such as Telegram, LinkedIn, and Viber. The target audience comprised specific international groups, primarily Russian, German, and English-speaking, of clinicians and scientific researchers. These participants ranged in their levels of expertise and experience, from beginners to veterans. The survey centered on a singular, pivotal question: “Do You Consent to the Use of Your Biological and Private Data for Training Machine Learning and AI Models?” Respondents had the option to choose from three responses: “Yes” and “No”. Results: The survey was conducted in January 2024. A total of 119 unique and verified individuals participated in the survey. The results revealed that only 50% of respondents (63 persons) expressed consent to provide their own data for the training of ML and AI models. Conclusion: In the development of ML and AI models, particularly open-source ones, it is crucial to ascertain whether participants are willing to provide their private data. While ML algorithms can transform the nature of data, it is important to remember that the primary owner of this data is the individual. Our findings show that in 50% of the cases, even participants from scientific research and clinical backgrounds – individuals typically accountable for ensuring data quality in AI and ML model development – do not consent to the use of their data in AI and ML settings. This highlights the need for more stringent consent processes and ethical considerations in the utilization of personal data in AI and ML research.

This paper provides an overview of the following three modeling approaches – Physics-based Modeling, Machine Learning, and Hybrid Modeling (Physics-based Modeling & Machine Learning combined), and addresses their applicability for multiphase flow simulations in specific use-cases. Physics-based modeling builds on well understood concepts in, e.g., thermodynamics, fluid dynamics, fluid modeling and optimization techniques. It requires deep domain knowledge as well as accurate fluid data and may incur significant computational cost. Machine Learning systems are based on learning algorithms, which find relationships between sensor data and output variables in a training dataset. The approach requires a good understanding of the learning algorithms and statistics. Small datasets or changes in operational conditions limit the suitability of this approach. Hybrid Models combine Physics with Machine Learning, and these models are on a sliding scale between pure Physics-based Models and pure Machine Learning Models. The individual use-case defines how the Hybrid Model is configured and where it sits on the sliding scale. Here, we will investigate three different use-cases: Physics supporting Machine Learning: Use physics to generate data to support training of machine learning models when data is sparse. Machine Learning supporting Physics: Use machine learning to provide additional input to physics modeling when available data does not suffice to achieve accurate numerical solutions. True Hybrid: Close the circle and use physics for feature engineering to improve the machine learning models, which then provide synthetic data as input to physics-based simulations. We observe that there is not one universal solution. On the one hand, flow simulations may address situations which are stochastic in nature and not deterministic. Here, a machine learning model based on physics-based simulations fits the purpose well. On the other hand, there are situations when machine learning can deduce relations between parameters such that you can provide additional input into physics-based models. One such example is to address the oil-water split at very high water cuts. By adopting the appropriate combination, the hybrid approach results in superior accuracy and offers the ability to address a broader range of applications than physics or machine learning alone.

Fatal accidents are a major issue hindering the wide acceptance of safety-critical systems that employ machine learning and deep learning models, such as automated driving vehicles. In order to use machine learning in a safety-critical system, it is necessary to demonstrate the safety and security of the system through engineering processes. However, thus far, no such widely accepted engineering concepts or frameworks have been established for these systems. The key to using a machine learning model in a deductively engineered system is decomposing the data-driven training of machine learning models into requirement, design, and verification, particularly for machine learning models used in safety-critical systems. Simultaneously, open problems and relevant technical fields are not organized in a manner that enables researchers to select a theme and work on it. In this study, we identify, classify, and explore the open problems in engineering (safety-critical) machine learning systems—that is, in terms of requirement, design, and verification of machine learning models and systems—as well as discuss related works and research directions, using automated driving vehicles as an example. Our results show that machine learning models are characterized by a lack of requirements specification, lack of design specification, lack of interpretability, and lack of robustness. We also perform a gap analysis on a conventional system quality standard SQuaRE with the characteristics of machine learning models to study quality models for machine learning systems. We find that a lack of requirements specification and lack of robustness have the greatest impact on conventional quality models.

This paper investigates the importance of explainability versus accuracy of machine learning (ML) models. We propose that greater task uncertainty makes people want to interact more with the ML model, which increases the importance of explainability relative to accuracy. We focus on the use of ML models for product cost estimation during new product development. The paper provides mixed-methods evidence on the trade-off between explainability and accuracy of ML models. Specifically, we find support for an inverse relationship between explainability and accuracy from the perspective of cost experts. We also find that the accurate but complex and less explainable ML model of gradient boosted regression (GBR) was preferred in only a few situations; mostly, the more basic, better explainable models of multiple linear regression (MLR) and case-based reasoning (CBR) were preferred, although these were less accurate. This suggests that lack of explainability can indeed be a major limitation for the application of ML models. Furthermore, we investigate specific characteristics that could increase task uncertainty and the importance of explainability in our context: project unpredictability, product cost granularity, predecessor product availability, target cost gap, and product development phase.

Accurate estimation of reference evapotranspiration (ETo) plays a vital role in irrigation and water resource planning. The Penman–Monteith method recommended by the Food and Agriculture Organization (FAO PM56) is widely used and considered a standard to calculate ETo. However, FAO PM56 cannot be used with limited meteorological variables, so it is compulsory to choose an alternative model for ETo estimation, which requires fewer variables. This study built ten machine learning (ML) models based on multi-function, neural network, and tree-based structure against the FAO PM56 method. For this purpose, time series temperature data on a monthly scale are only used to train ML models. The developed ML models were applied to estimate ETo at different test stations and the obtained results were compared with the FAO PM56 method to verify and validate their performance in ETo estimation for the selected stations. In addition, multiple statistical indicators, including root-mean-square error (RMSE), coefficient of determination (R2), mean absolute error (MAE), Nash–Sutcliffe efficiency (NSE), and correlation coefficient (r) were calculated to compare the performance of each ML model on ETo estimation. Among the applied ML models, the ETo tree boost (TB) ML model outperformed the other ML models in estimating ETo in diverse climatic conditions based on statistical indicators (R2, NSE, r, RMSE, and MAE). Moreover, the observed R2, NSE, and r were the highest for the TB ML model, while RMSE and MAE were found to be the lowest at the study sites compared to other applied ML models. Lastly, ETo point data yielded from the TB ML model was used in an interpolation process to create monthly and annual ETo maps. Based on the ETo maps, this study suggests mainly a focus on areas with high ETo values and proper irrigation scheduling of crops to ensure water sustainability.

The objective of this document is to identify the most accurate machine learning (ML) models for predicting the value of a real estate property, based on a systematic literature review (SLR). It was conducted on research published between 2022 and 2023 that analyzed the accuracy of ML models in real estate valuation. Information was extracted regarding the ML models used, the databases employed, and the models highlighted for their accuracy. A variety of ML models used in real estate valuation were identified, including Random Forest (RF), XGBoost, Gradient Boosting Machine (GBM), Linear Regression (LR), and Lasso Regression. RF and LR models stood out as the most accurate in the analyzed research, showing that the accuracy of ML models varies depending on the database, property characteristics, and valuation context. It was concluded that ML models like RF and LR are promising tools for improving the accuracy of real estate valuation. Additionally, the choice of the best ML model depends on factors such as the database, property characteristics, and specific valuation objectives. Further research is needed on the application of ML in real estate valuation, considering aspects such as model interpretability and the real estate market context.

Applying machine learning (ML) to sensitive domains requires privacy protection of the underlying training data through formal privacy frameworks, such as differential privacy (DP). Yet, usually, the privacy of the training data comes at the cost of the resulting ML models' utility. One reason for this is that DP uses one uniform privacy budget epsilon for all training data points, which has to align with the strictest privacy requirement encountered among all data holders. In practice, different data holders have different privacy requirements and data points of data holders with lower requirements can contribute more information to the training process of the ML models. To account for this need, we propose two novel methods based on the Private Aggregation of Teacher Ensembles (PATE) framework to support the training of ML models with individualized privacy guarantees. We formally describe the methods, provide a theoretical analysis of their privacy bounds, and experimentally evaluate their effect on the final model's utility using the MNIST, SVHN, and Adult income datasets. Our empirical results show that the individualized privacy methods yield ML models of higher accuracy than the non-individualized baseline. Thereby, we improve the privacy-utility trade-off in scenarios in which different data holders consent to contribute their sensitive data at different individual privacy levels.

Federated learning (FL) has been vigorously promoted in wireless edge networks as it fosters collaborative training of machine learning (ML) models while preserving individual user privacy and data security. In conventional FL, user equipment (UE) and an aggregator can collaboratively train shared global ML models by transmitting interactive ML models. In wireless edge networks, heterogeneity of multi-dimensional resources (e.g., computing and communication resources) used to train and transmit FL models may introduce stragglers, characterized by a slow update and/or transmission of local models. The stragglers can significantly degrade learning performance of FL, as the slowest participating UE can dramatically slow down the entire convergence. In this paper, to alleviate the negative impact of stragglers, we propose a dynamic straggler-aware clustering based FL (FeDSC) mechanism via adaptive UE clustering. Specifically, we first group participating UEs into multiple clusters based on their available computing and wireless resources. Then, we propose an adaptive clustering scheme to synchronously update the cluster aggregation model. Meanwhile, an edge server performs global aggregation of different cluster models in an asynchronous manner. Finally, we theoretically demonstrate the convergence of our proposed mechanism via numerical results. Numerical results show that our proposed mechanism can effectively reduce training time and wireless bandwidth consumption, while improving training efficiency and guaranteeing learning accuracy.

<p>Background: The prediction of aneurysm treatment outcomes can help to optimize the treatment strategies. Machine learning (ML) has shown positive results in many clinical areas. However, the development of such models requires expertise in ML, which is not an easy task for surgeons.</p><p>Objectives: The recently emerged automated machine learning (AutoML) has shown promise in making ML more accessible to non-computer experts. We aimed to evaluate the feasibility of applying AutoML to develop the ML models for treatment outcome prediction.</p><p>Methods: The patients with aneurysms treated by endovascular treatment were prospectively recruited from 2016 to 2020. Treatment was considered successful if angiographic complete occlusion was achieved at follow-up. A statistical prediction model was developed using multivariate logistic regression. In addition, two ML models were developed. One was developed manually and the other was developed by AutoML. Three models were compared based on their area under the precision-recall curve (AUPRC) and area under the receiver operating characteristic curve (AUROC).</p><p>Results: The aneurysm size, stent-assisted coiling (SAC), and posterior circulation were the three significant and independent variables associated with treatment outcome. The statistical model showed an AUPRC of 0.432 and AUROC of 0.745. The conventional manually trained ML model showed an improved AUPRC of 0.545 and AUROC of 0.781. The AutoML derived ML model showed the best performance with AUPRC of 0.632 and AUROC of 0.832, significantly better than the other two models.</p><p>Conclusions: This study demonstrated the feasibility of using AutoML to develop a high-quality ML model, which may outperform the statistical model and manually derived ML models. AutoML could be a useful tool that makes ML more accessible to the clinical researchers.</p>

Reconstructing the emergence of the modern continental crust is crucial to understand the evolution of the crust, the onset of plate tectonics, elemental cycling, and long-term climate. However, it remains highly contentious about when and how the major subaerial continental crust emerged over time. Here, we used a machine learning (ML) model (XGBoost) to reveal the exposed history of continental crust over the last 3.8 billion years ago (Ga). First, we compiled ~10,000 modern subaerial or submarine basalts with major and trace elements to train the ML model. Then, the trained ML model (with resampling) was utilized to predict and calculate the mean proportions of subaerially erupted continental basaltic rocks since 3.8 Ga. The result suggested that the subaerial proportion only reached about 50% at ~2.5 Ga, indicating the exposure of the continental crust was far from the present-day level at the end of the Archean era. On the other hand, since ~1.8 Ga, the subaerial proportion of the continental crust exhibited a dynamic balance between ~60% and 80%, reaching the present-day level.

HADA: An automated tool for hardware dimensioning of AI applications

Federated learning enables privacy-preserving and data-efficient dimension prediction and part qualification across additive manufacturing factories

The pooled cohort equations (PCE) predict atherosclerotic cardiovascular disease (ASCVD) risk in patients with characteristics within prespecified ranges and has uncertain performance among Asians or Hispanics. It is unknown if machine learning (ML) models can improve ASCVD risk prediction across broader diverse, real-world populations. We developed ML models for ASCVD risk prediction for multi-ethnic patients using an electronic health record (EHR) database from Northern California. Our cohort included patients aged 18 years or older with no prior CVD and not on statins at baseline (n = 262,923), stratified by PCE-eligible (n = 131,721) or PCE-ineligible patients based on missing or out-of-range variables. We trained ML models [logistic regression with L2 penalty and L1 lasso penalty, random forest, gradient boosting machine (GBM), extreme gradient boosting] and determined 5-year ASCVD risk prediction, including with and without incorporation of additional EHR variables, and in Asian and Hispanic subgroups. A total of 4309 patients had ASCVD events, with 2077 in PCE-ineligible patients. GBM performance in the full cohort, including PCE-ineligible patients (area under receiver-operating characteristic curve (AUC) 0.835, 95% confidence interval (CI): 0.825–0.846), was significantly better than that of the PCE in the PCE-eligible cohort (AUC 0.775, 95% CI: 0.755–0.794). Among patients aged 40–79, GBM performed similarly before (AUC 0.784, 95% CI: 0.759–0.808) and after (AUC 0.790, 95% CI: 0.765–0.814) incorporating additional EHR data. Overall, ML models achieved comparable or improved performance compared to the PCE while allowing risk discrimination in a larger group of patients including PCE-ineligible patients. EHR-trained ML models may help bridge important gaps in ASCVD risk prediction.

A machine learning model exploring creep performance of dental composites.

Batteries, recognized as effective energy storage solutions, are considered the main facilitators of the world-wide transition towards clean and renewable energy sources. Among different types of batteries, lithium-ion (Li-ion) variants offer higher energy densities and relatively longer life spans when compared to other types. Nonetheless, a primary concern with these batteries is their lifetime. Batteries undergo various degradation mechanisms under storage and use, significantly impacting their lifespan. To this end, it is crucial to predict the degradation and lifetime of Li-ion batteries under given conditions.Researchers use three main methodologies to perform battery health diagnostics and to predict the lifetime of batteries. One approach revolves around using mechanistic, first-principle electrochemical models, also known as physics-based models. If equipped with proper thermodynamic theories, such models can show promising capabilities; however, holistic degradation prediction with these models is still challenging due to the computational complexities and the multitude of parameters that need to be fine-tuned in this approach. The other common strategy is to use empirical models to predict battery degradation. These models generally entail fewer parameters to be identified and are computationally less intensive to solve. Nonetheless, empirical models can suffer from the accuracy point-of-view as they are constrained to predict degradation trends introduced by certain degradation modes. Another common technique in battery life prediction is to utilize purely data-driven methods, such as machine learning (ML) algorithms, which also have shown promising results in the literature on rapid health predictions. However, these methods require large volumes of experimental data for training and testing ML models to ensure accuracy. In addition, data-driven methods are likely to extrapolate poorly to conditions beyond their training data and are indifferent towards the underlying degradation mechanisms. Recently, physics-informed machine learning (PI-ML) methods have garnered significant attention. They integrate physics-based or empirical models (developed based on physics) with a data-driven approach and allow one to train ML models on a smaller set of experimental data.To the best of the authors’ knowledge, the performance comparison between first-principle and empirical models when integrated within PI-ML remains unclear. Therefore, in this work, we aim to compare these two models when applied to prognostics (capacity forecasting and remaining useful life prediction) of a set of Li-ion batteries. To perform this study, we generate aging data for 40 Li-ion coin cells cycled under randomized conditions. Each cell undergoes a three-step charging stage followed by a two-step discharge stage. After obtaining the aging data, we will develop two PI-ML models, one equipped with a physics-based model and another with a set of empirical models. Both PI-ML models in this work will follow the sequential integration approach, where the training data for the final PI-ML model come from both experimental and computational data, the latter of which are obtained from the physics-based or empirical models. The parameters for the physics-based and empirical models are identified from another set of experimental data. Finally, the PI-ML models will be tested with experimental data obtained at different cycling conditions. The data flow for the sequential architecture of PI-ML is shown in the attached figure. This comparative study will help identify the performance of physics-based and empirical models when integrated into PI-ML. The main performance metric considered in this work is each model’s ability to extrapolate beyond the experimental training data set, hence aiding the final PI-ML model in generalizing to conditions not covered by its experimental training data set. Figure 1